Note: This is test shard 4 of 8. [==========] Running 16 tests from 3 test cases. [----------] Global test environment set-up. [----------] 14 tests from ClientTest [ RUN ] ClientTest.TestMasterDown WARNING: Logging before InitGoogleLogging() is written to STDERR I0609 06:56:44.715471 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:34767 I0609 06:56:44.736644 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 89016us I0609 06:56:44.737121 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:44.737287 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:44.737612 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:44.737787 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:44.741210 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterDown.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "575cfd9b5a7e432a9d45b1299064bec9" format_stamp: "Formatted at 2019-06-09 06:56:44 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:44.753020 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.011s user 0.009s sys 0.000s I0609 06:56:44.754123 1119 env_posix.cc:1676] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0609 06:56:44.754717 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:44.762215 1119 fs_manager.cc:419] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0609 06:56:44.762374 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterDown.1560063404629499-1119/minicluster-data/master-0-root uuid: "575cfd9b5a7e432a9d45b1299064bec9" format_stamp: "Formatted at 2019-06-09 06:56:44 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:44.762639 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterDown.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterDown.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterDown.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) W0609 06:56:45.884382 1119 thread.cc:599] rpc reactor (reactor) Time spent creating pthread: real 1.091s user 0.419s sys 0.672s W0609 06:56:45.884601 1119 thread.cc:566] rpc reactor (reactor) Time spent starting thread: real 1.091s user 0.419s sys 0.672s I0609 06:56:45.960669 1119 env_posix.cc:1676] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0609 06:56:46.029044 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:34767 I0609 06:56:46.031610 1119 webserver.cc:374] Webserver started at http://127.1.23.254:46175/ using document root and password file I0609 06:56:46.034626 1180 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:46.052233 1180 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9: Bootstrap starting. I0609 06:56:46.055905 1180 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9: No blocks or log segments found. Creating new log. I0609 06:56:46.057595 1180 log.cc:525] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9: Log is configured to *not* fsync() on all Append() calls I0609 06:56:46.060467 1180 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9: No bootstrap required, opened a new log I0609 06:56:46.075260 1180 raft_consensus.cc:340] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575cfd9b5a7e432a9d45b1299064bec9" member_type: VOTER } I0609 06:56:46.075713 1180 raft_consensus.cc:366] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:46.075930 1180 raft_consensus.cc:705] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575cfd9b5a7e432a9d45b1299064bec9, State: Initialized, Role: FOLLOWER I0609 06:56:46.076565 1180 consensus_queue.cc:229] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575cfd9b5a7e432a9d45b1299064bec9" member_type: VOTER } I0609 06:56:46.077029 1180 raft_consensus.cc:380] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:46.077273 1180 raft_consensus.cc:472] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:46.077603 1180 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:46.080667 1180 raft_consensus.cc:494] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575cfd9b5a7e432a9d45b1299064bec9" member_type: VOTER } I0609 06:56:46.081248 1180 leader_election.cc:310] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 575cfd9b5a7e432a9d45b1299064bec9; no voters: I0609 06:56:46.083317 1182 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:46.084645 1180 leader_election.cc:296] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:46.158927 1182 raft_consensus.cc:667] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [term 1 LEADER]: Becoming Leader. State: Replica: 575cfd9b5a7e432a9d45b1299064bec9, State: Running, Role: LEADER I0609 06:56:46.159477 1182 consensus_queue.cc:206] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575cfd9b5a7e432a9d45b1299064bec9" member_type: VOTER } I0609 06:56:46.164420 1180 sys_catalog.cc:338] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "575cfd9b5a7e432a9d45b1299064bec9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575cfd9b5a7e432a9d45b1299064bec9" member_type: VOTER } } I0609 06:56:46.164357 1184 sys_catalog.cc:338] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "575cfd9b5a7e432a9d45b1299064bec9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575cfd9b5a7e432a9d45b1299064bec9" member_type: VOTER } } I0609 06:56:46.165091 1184 sys_catalog.cc:341] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [sys.catalog]: This master's current role is: LEADER I0609 06:56:46.165046 1180 sys_catalog.cc:341] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [sys.catalog]: This master's current role is: LEADER I0609 06:56:46.164299 1183 sys_catalog.cc:338] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 575cfd9b5a7e432a9d45b1299064bec9. Latest consensus state: current_term: 1 leader_uuid: "575cfd9b5a7e432a9d45b1299064bec9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575cfd9b5a7e432a9d45b1299064bec9" member_type: VOTER } } I0609 06:56:46.165477 1183 sys_catalog.cc:341] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [sys.catalog]: This master's current role is: LEADER I0609 06:56:46.168596 1180 sys_catalog.cc:426] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:46.169368 1187 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:46.170460 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:46.179515 1187 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:46.274646 1187 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:46.276348 1187 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:46.293720 1187 catalog_manager.cc:4293] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9: Generated new TSK 0 I0609 06:56:46.335104 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 90016us I0609 06:56:46.336520 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:46.336738 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:46.337033 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:46.337234 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:46.338974 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterDown.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "62db0506113d4f9589a07e0cd364234b" format_stamp: "Formatted at 2019-06-09 06:56:46 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:46.358861 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.019s user 0.017s sys 0.004s I0609 06:56:46.360915 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:46.375488 1119 fs_manager.cc:419] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s I0609 06:56:46.375705 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterDown.1560063404629499-1119/minicluster-data/ts-0-root uuid: "62db0506113d4f9589a07e0cd364234b" format_stamp: "Formatted at 2019-06-09 06:56:46 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:46.376029 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterDown.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterDown.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterDown.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:46.430977 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:46.431268 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:46.568553 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:37861 I0609 06:56:46.571106 1119 webserver.cc:374] Webserver started at http://127.1.23.193:35419/ using document root and password file I0609 06:56:46.592844 1255 heartbeater.cc:324] Connected to a master server at 127.1.23.254:34767 I0609 06:56:46.593256 1255 heartbeater.cc:405] Registering TS with master... I0609 06:56:46.594002 1255 heartbeater.cc:454] Master 127.1.23.254:34767 requested a full tablet report, sending... I0609 06:56:46.596271 1147 ts_manager.cc:161] Registered new tserver with Master: 62db0506113d4f9589a07e0cd364234b (127.1.23.193:37861) I0609 06:56:46.596683 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.022159296s I0609 06:56:46.599649 1147 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53128 I0609 06:56:46.612650 1255 heartbeater.cc:446] Master 127.1.23.254:34767 was elected leader, sending a full tablet report... I0609 06:56:46.641985 1147 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53130: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:46.687839 1221 tablet_service.cc:1016] Processing CreateTablet for tablet 44165c667eae4380868247fea6c55c7a (table=client-testtb [id=f573193e52ce4230816a6cc68590dff3]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:46.687116 1222 tablet_service.cc:1016] Processing CreateTablet for tablet be2443e0b85446128a6f59df07fff719 (table=client-testtb [id=f573193e52ce4230816a6cc68590dff3]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:46.689060 1221 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 44165c667eae4380868247fea6c55c7a. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:46.689524 1222 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet be2443e0b85446128a6f59df07fff719. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:46.701984 1267 tablet_bootstrap.cc:439] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b: Bootstrap starting. I0609 06:56:46.706120 1267 tablet_bootstrap.cc:590] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b: No blocks or log segments found. Creating new log. I0609 06:56:46.710381 1267 tablet_bootstrap.cc:439] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b: No bootstrap required, opened a new log I0609 06:56:46.710762 1267 ts_tablet_manager.cc:1054] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s I0609 06:56:46.712625 1267 raft_consensus.cc:340] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62db0506113d4f9589a07e0cd364234b" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 37861 } } I0609 06:56:46.713014 1267 raft_consensus.cc:366] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:46.713279 1267 raft_consensus.cc:705] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62db0506113d4f9589a07e0cd364234b, State: Initialized, Role: FOLLOWER I0609 06:56:46.713853 1267 consensus_queue.cc:229] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62db0506113d4f9589a07e0cd364234b" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 37861 } } I0609 06:56:46.714256 1267 raft_consensus.cc:380] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:46.714429 1267 raft_consensus.cc:472] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:46.714661 1267 raft_consensus.cc:2890] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:46.717932 1267 raft_consensus.cc:494] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62db0506113d4f9589a07e0cd364234b" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 37861 } } I0609 06:56:46.718484 1267 leader_election.cc:310] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62db0506113d4f9589a07e0cd364234b; no voters: I0609 06:56:46.720495 1267 leader_election.cc:296] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:46.721094 1269 raft_consensus.cc:2644] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:46.723748 1267 ts_tablet_manager.cc:1082] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s I0609 06:56:46.724567 1267 tablet_bootstrap.cc:439] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b: Bootstrap starting. I0609 06:56:46.725049 1269 raft_consensus.cc:667] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [term 1 LEADER]: Becoming Leader. State: Replica: 62db0506113d4f9589a07e0cd364234b, State: Running, Role: LEADER I0609 06:56:46.725594 1269 consensus_queue.cc:206] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62db0506113d4f9589a07e0cd364234b" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 37861 } } I0609 06:56:46.727262 1267 tablet_bootstrap.cc:590] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b: No blocks or log segments found. Creating new log. I0609 06:56:46.734481 1146 catalog_manager.cc:3989] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b reported cstate change: term changed from 0 to 1, leader changed from to 62db0506113d4f9589a07e0cd364234b (127.1.23.193). New cstate: current_term: 1 leader_uuid: "62db0506113d4f9589a07e0cd364234b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62db0506113d4f9589a07e0cd364234b" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 37861 } health_report { overall_health: HEALTHY } } } I0609 06:56:46.740352 1267 tablet_bootstrap.cc:439] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b: No bootstrap required, opened a new log I0609 06:56:46.740785 1267 ts_tablet_manager.cc:1054] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.012s I0609 06:56:46.744052 1267 raft_consensus.cc:340] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62db0506113d4f9589a07e0cd364234b" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 37861 } } I0609 06:56:46.744421 1267 raft_consensus.cc:366] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:46.744603 1267 raft_consensus.cc:705] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62db0506113d4f9589a07e0cd364234b, State: Initialized, Role: FOLLOWER I0609 06:56:46.745081 1267 consensus_queue.cc:229] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62db0506113d4f9589a07e0cd364234b" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 37861 } } I0609 06:56:46.745575 1267 raft_consensus.cc:380] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:46.745762 1267 raft_consensus.cc:472] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:46.746001 1267 raft_consensus.cc:2890] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:46.749783 1267 raft_consensus.cc:494] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62db0506113d4f9589a07e0cd364234b" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 37861 } } I0609 06:56:46.750218 1267 leader_election.cc:310] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62db0506113d4f9589a07e0cd364234b; no voters: I0609 06:56:46.750692 1267 leader_election.cc:296] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:46.750844 1269 raft_consensus.cc:2644] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:46.751710 1267 ts_tablet_manager.cc:1082] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s I0609 06:56:46.751957 1269 raft_consensus.cc:667] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [term 1 LEADER]: Becoming Leader. State: Replica: 62db0506113d4f9589a07e0cd364234b, State: Running, Role: LEADER I0609 06:56:46.752470 1269 consensus_queue.cc:206] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62db0506113d4f9589a07e0cd364234b" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 37861 } } I0609 06:56:46.757138 1146 catalog_manager.cc:3989] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b reported cstate change: term changed from 0 to 1, leader changed from to 62db0506113d4f9589a07e0cd364234b (127.1.23.193). New cstate: current_term: 1 leader_uuid: "62db0506113d4f9589a07e0cd364234b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62db0506113d4f9589a07e0cd364234b" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 37861 } health_report { overall_health: HEALTHY } } } I0609 06:56:46.769556 1119 master.cc:247] Master@127.1.23.254:34767 shutting down... I0609 06:56:46.777088 1129 messenger.cc:381] Service unavailable: service kudu.master.MasterService not registered on Master W0609 06:56:46.779047 1255 heartbeater.cc:566] Failed to heartbeat to 127.1.23.254:34767 (0 consecutive failures): Remote error: Failed to send heartbeat to master: Service unavailable: service kudu.master.MasterService not registered on Master I0609 06:56:46.790421 1129 messenger.cc:381] Service unavailable: service kudu.master.MasterService not registered on Master I0609 06:56:46.802285 1129 messenger.cc:381] Service unavailable: service kudu.master.MasterService not registered on Master I0609 06:56:46.814276 1129 messenger.cc:381] Service unavailable: service kudu.master.MasterService not registered on Master I0609 06:56:46.821084 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:46.821642 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:46.821993 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 575cfd9b5a7e432a9d45b1299064bec9: stopping tablet replica I0609 06:56:46.827476 1129 messenger.cc:381] Service unavailable: service kudu.master.MasterService not registered on Master I0609 06:56:46.867189 1119 master.cc:258] Master@127.1.23.254:34767 shutdown complete. I0609 06:56:46.877602 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:46.946616 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:46.947199 1119 tablet_replica.cc:265] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b: stopping tablet replica I0609 06:56:46.947769 1119 raft_consensus.cc:2147] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:46.948132 1119 raft_consensus.cc:2174] T be2443e0b85446128a6f59df07fff719 P 62db0506113d4f9589a07e0cd364234b [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:46.951473 1119 tablet_replica.cc:265] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b: stopping tablet replica I0609 06:56:46.951877 1119 raft_consensus.cc:2147] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:46.952209 1119 raft_consensus.cc:2174] T 44165c667eae4380868247fea6c55c7a P 62db0506113d4f9589a07e0cd364234b [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:46.985324 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:46.985536 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:46.985947 1119 tablet_server.cc:169] TabletServer shutdown complete. [ OK ] ClientTest.TestMasterDown (2292 ms) [ RUN ] ClientTest.TestProjectInvalidColumn I0609 06:56:47.006089 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:45235 I0609 06:56:47.029554 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 90516us I0609 06:56:47.030016 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:47.030211 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:47.030519 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:47.030707 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:47.031916 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestProjectInvalidColumn.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "d9b1c9252ced4de2ab1575f6e2c43e29" format_stamp: "Formatted at 2019-06-09 06:56:47 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:47.046028 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.014s user 0.016s sys 0.001s I0609 06:56:47.047878 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:47.060570 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:47.060782 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestProjectInvalidColumn.1560063404629499-1119/minicluster-data/master-0-root uuid: "d9b1c9252ced4de2ab1575f6e2c43e29" format_stamp: "Formatted at 2019-06-09 06:56:47 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:47.061050 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestProjectInvalidColumn.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestProjectInvalidColumn.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestProjectInvalidColumn.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:47.186825 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:45235 I0609 06:56:47.189910 1119 webserver.cc:374] Webserver started at http://127.1.23.254:35595/ using document root and password file I0609 06:56:47.194504 1330 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:47.206423 1330 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29: Bootstrap starting. I0609 06:56:47.208361 1330 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29: No blocks or log segments found. Creating new log. I0609 06:56:47.218307 1330 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29: No bootstrap required, opened a new log I0609 06:56:47.219729 1330 raft_consensus.cc:340] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b1c9252ced4de2ab1575f6e2c43e29" member_type: VOTER } I0609 06:56:47.220070 1330 raft_consensus.cc:366] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:47.220315 1330 raft_consensus.cc:705] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b1c9252ced4de2ab1575f6e2c43e29, State: Initialized, Role: FOLLOWER I0609 06:56:47.220810 1330 consensus_queue.cc:229] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b1c9252ced4de2ab1575f6e2c43e29" member_type: VOTER } I0609 06:56:47.221201 1330 raft_consensus.cc:380] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:47.221441 1330 raft_consensus.cc:472] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:47.221699 1330 raft_consensus.cc:2890] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:47.224858 1330 raft_consensus.cc:494] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b1c9252ced4de2ab1575f6e2c43e29" member_type: VOTER } I0609 06:56:47.225250 1330 leader_election.cc:310] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9b1c9252ced4de2ab1575f6e2c43e29; no voters: I0609 06:56:47.229629 1330 leader_election.cc:296] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:47.229781 1332 raft_consensus.cc:2644] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:47.231065 1332 raft_consensus.cc:667] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [term 1 LEADER]: Becoming Leader. State: Replica: d9b1c9252ced4de2ab1575f6e2c43e29, State: Running, Role: LEADER I0609 06:56:47.231571 1332 consensus_queue.cc:206] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b1c9252ced4de2ab1575f6e2c43e29" member_type: VOTER } I0609 06:56:47.234452 1334 sys_catalog.cc:338] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9b1c9252ced4de2ab1575f6e2c43e29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b1c9252ced4de2ab1575f6e2c43e29" member_type: VOTER } } I0609 06:56:47.235030 1334 sys_catalog.cc:341] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [sys.catalog]: This master's current role is: LEADER I0609 06:56:47.235502 1333 sys_catalog.cc:338] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9b1c9252ced4de2ab1575f6e2c43e29. Latest consensus state: current_term: 1 leader_uuid: "d9b1c9252ced4de2ab1575f6e2c43e29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b1c9252ced4de2ab1575f6e2c43e29" member_type: VOTER } } I0609 06:56:47.236315 1333 sys_catalog.cc:341] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [sys.catalog]: This master's current role is: LEADER I0609 06:56:47.236057 1330 sys_catalog.cc:338] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "d9b1c9252ced4de2ab1575f6e2c43e29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b1c9252ced4de2ab1575f6e2c43e29" member_type: VOTER } } I0609 06:56:47.236690 1330 sys_catalog.cc:341] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [sys.catalog]: This master's current role is: LEADER I0609 06:56:47.239235 1330 sys_catalog.cc:426] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:47.240377 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 W0609 06:56:47.241971 1338 catalog_manager.cc:1114] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29: acquiring CA information for follower catalog manager: Not found: root CA entry not found W0609 06:56:47.242152 1338 catalog_manager.cc:597] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry I0609 06:56:47.245481 1336 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:47.247738 1336 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:47.284121 1336 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:47.285559 1336 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:47.293227 1336 catalog_manager.cc:4293] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29: Generated new TSK 0 I0609 06:56:47.318584 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 90516us I0609 06:56:47.318925 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:47.319070 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:47.319284 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:47.319401 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:47.320322 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestProjectInvalidColumn.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "0b0d461ecc4847069a52d4969c9bea54" format_stamp: "Formatted at 2019-06-09 06:56:47 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:47.330358 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.010s user 0.011s sys 0.000s I0609 06:56:47.331657 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:47.342205 1119 fs_manager.cc:419] Time spent opening block manager: real 0.005s user 0.001s sys 0.004s I0609 06:56:47.342372 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestProjectInvalidColumn.1560063404629499-1119/minicluster-data/ts-0-root uuid: "0b0d461ecc4847069a52d4969c9bea54" format_stamp: "Formatted at 2019-06-09 06:56:47 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:47.342623 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestProjectInvalidColumn.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestProjectInvalidColumn.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestProjectInvalidColumn.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:47.422056 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:47.422201 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:47.537636 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:44503 I0609 06:56:47.539391 1119 webserver.cc:374] Webserver started at http://127.1.23.193:44009/ using document root and password file I0609 06:56:47.569241 1404 heartbeater.cc:324] Connected to a master server at 127.1.23.254:45235 I0609 06:56:47.569545 1404 heartbeater.cc:405] Registering TS with master... I0609 06:56:47.569967 1404 heartbeater.cc:454] Master 127.1.23.254:45235 requested a full tablet report, sending... I0609 06:56:47.571477 1297 ts_manager.cc:161] Registered new tserver with Master: 0b0d461ecc4847069a52d4969c9bea54 (127.1.23.193:44503) I0609 06:56:47.572008 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.021814238s I0609 06:56:47.573119 1297 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58552 I0609 06:56:47.586081 1404 heartbeater.cc:446] Master 127.1.23.254:45235 was elected leader, sending a full tablet report... I0609 06:56:47.601953 1297 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58554: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:47.630028 1371 tablet_service.cc:1016] Processing CreateTablet for tablet e3ce89a2f6484512a12922471c794f74 (table=client-testtb [id=074a9e1a7b69419a81db13a68be55b0f]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:47.630342 1370 tablet_service.cc:1016] Processing CreateTablet for tablet 0b57d9438b9f42008bfaf6dc9d7d2aef (table=client-testtb [id=074a9e1a7b69419a81db13a68be55b0f]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:47.630998 1371 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet e3ce89a2f6484512a12922471c794f74. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:47.631470 1370 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 0b57d9438b9f42008bfaf6dc9d7d2aef. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:47.643045 1416 tablet_bootstrap.cc:439] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54: Bootstrap starting. I0609 06:56:47.645956 1416 tablet_bootstrap.cc:590] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54: No blocks or log segments found. Creating new log. I0609 06:56:47.649677 1416 tablet_bootstrap.cc:439] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54: No bootstrap required, opened a new log I0609 06:56:47.649996 1416 ts_tablet_manager.cc:1054] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s I0609 06:56:47.651480 1416 raft_consensus.cc:340] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d461ecc4847069a52d4969c9bea54" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44503 } } I0609 06:56:47.652060 1416 raft_consensus.cc:366] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:47.652297 1416 raft_consensus.cc:705] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0d461ecc4847069a52d4969c9bea54, State: Initialized, Role: FOLLOWER I0609 06:56:47.652840 1416 consensus_queue.cc:229] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d461ecc4847069a52d4969c9bea54" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44503 } } I0609 06:56:47.653256 1416 raft_consensus.cc:380] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:47.653486 1416 raft_consensus.cc:472] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:47.653733 1416 raft_consensus.cc:2890] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:47.657773 1416 raft_consensus.cc:494] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d461ecc4847069a52d4969c9bea54" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44503 } } I0609 06:56:47.658195 1416 leader_election.cc:310] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b0d461ecc4847069a52d4969c9bea54; no voters: I0609 06:56:47.659940 1418 raft_consensus.cc:2644] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:47.661006 1416 leader_election.cc:296] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:47.663357 1418 raft_consensus.cc:667] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [term 1 LEADER]: Becoming Leader. State: Replica: 0b0d461ecc4847069a52d4969c9bea54, State: Running, Role: LEADER I0609 06:56:47.663969 1416 ts_tablet_manager.cc:1082] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54: Time spent starting tablet: real 0.014s user 0.008s sys 0.003s I0609 06:56:47.663936 1418 consensus_queue.cc:206] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d461ecc4847069a52d4969c9bea54" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44503 } } I0609 06:56:47.664908 1416 tablet_bootstrap.cc:439] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54: Bootstrap starting. I0609 06:56:47.667466 1416 tablet_bootstrap.cc:590] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54: No blocks or log segments found. Creating new log. I0609 06:56:47.672322 1296 catalog_manager.cc:3989] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 reported cstate change: term changed from 0 to 1, leader changed from to 0b0d461ecc4847069a52d4969c9bea54 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "0b0d461ecc4847069a52d4969c9bea54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d461ecc4847069a52d4969c9bea54" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44503 } health_report { overall_health: HEALTHY } } } I0609 06:56:47.681460 1416 tablet_bootstrap.cc:439] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54: No bootstrap required, opened a new log I0609 06:56:47.681782 1416 ts_tablet_manager.cc:1054] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.003s I0609 06:56:47.683089 1416 raft_consensus.cc:340] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d461ecc4847069a52d4969c9bea54" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44503 } } I0609 06:56:47.683478 1416 raft_consensus.cc:366] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:47.683689 1416 raft_consensus.cc:705] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0d461ecc4847069a52d4969c9bea54, State: Initialized, Role: FOLLOWER I0609 06:56:47.684391 1416 consensus_queue.cc:229] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d461ecc4847069a52d4969c9bea54" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44503 } } I0609 06:56:47.684835 1416 raft_consensus.cc:380] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:47.685025 1416 raft_consensus.cc:472] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:47.685277 1416 raft_consensus.cc:2890] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:47.688498 1416 raft_consensus.cc:494] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d461ecc4847069a52d4969c9bea54" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44503 } } I0609 06:56:47.688920 1416 leader_election.cc:310] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b0d461ecc4847069a52d4969c9bea54; no voters: I0609 06:56:47.689501 1416 leader_election.cc:296] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:47.689637 1418 raft_consensus.cc:2644] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:47.690151 1418 raft_consensus.cc:667] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [term 1 LEADER]: Becoming Leader. State: Replica: 0b0d461ecc4847069a52d4969c9bea54, State: Running, Role: LEADER I0609 06:56:47.690659 1416 ts_tablet_manager.cc:1082] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s I0609 06:56:47.690500 1418 consensus_queue.cc:206] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d461ecc4847069a52d4969c9bea54" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44503 } } I0609 06:56:47.695871 1296 catalog_manager.cc:3989] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 reported cstate change: term changed from 0 to 1, leader changed from to 0b0d461ecc4847069a52d4969c9bea54 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "0b0d461ecc4847069a52d4969c9bea54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0d461ecc4847069a52d4969c9bea54" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44503 } health_report { overall_health: HEALTHY } } } I0609 06:56:47.713924 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:47.738062 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:47.738459 1119 tablet_replica.cc:265] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54: stopping tablet replica I0609 06:56:47.738862 1119 raft_consensus.cc:2147] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:47.739145 1119 raft_consensus.cc:2174] T 0b57d9438b9f42008bfaf6dc9d7d2aef P 0b0d461ecc4847069a52d4969c9bea54 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:47.741766 1119 tablet_replica.cc:265] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54: stopping tablet replica I0609 06:56:47.742012 1119 raft_consensus.cc:2147] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:47.742223 1119 raft_consensus.cc:2174] T e3ce89a2f6484512a12922471c794f74 P 0b0d461ecc4847069a52d4969c9bea54 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:47.763837 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:47.763981 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:47.764242 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:47.770030 1119 master.cc:247] Master@127.1.23.254:45235 shutting down... I0609 06:56:47.784641 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:47.785043 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:47.785493 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P d9b1c9252ced4de2ab1575f6e2c43e29: stopping tablet replica I0609 06:56:47.796497 1119 master.cc:258] Master@127.1.23.254:45235 shutdown complete. [ OK ] ClientTest.TestProjectInvalidColumn (805 ms) [ RUN ] ClientTest.TestNonFaultTolerantScannerExpired I0609 06:56:47.810180 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:40485 I0609 06:56:47.822044 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 90516us I0609 06:56:47.822340 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:47.822466 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:47.822643 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:47.822752 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:47.823618 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "9c563af166844998a91fd07b50277c01" format_stamp: "Formatted at 2019-06-09 06:56:47 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:47.832115 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.006s sys 0.003s I0609 06:56:47.833410 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:47.842591 1119 fs_manager.cc:419] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s I0609 06:56:47.842782 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/master-0-root uuid: "9c563af166844998a91fd07b50277c01" format_stamp: "Formatted at 2019-06-09 06:56:47 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:47.843040 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:48.041821 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:40485 I0609 06:56:48.044351 1119 webserver.cc:374] Webserver started at http://127.1.23.254:36273/ using document root and password file I0609 06:56:48.050153 1479 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:48.060896 1479 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01: Bootstrap starting. I0609 06:56:48.075073 1479 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01: No blocks or log segments found. Creating new log. I0609 06:56:48.078115 1479 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01: No bootstrap required, opened a new log I0609 06:56:48.079499 1479 raft_consensus.cc:340] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c563af166844998a91fd07b50277c01" member_type: VOTER } I0609 06:56:48.079838 1479 raft_consensus.cc:366] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:48.080034 1479 raft_consensus.cc:705] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c563af166844998a91fd07b50277c01, State: Initialized, Role: FOLLOWER I0609 06:56:48.080515 1479 consensus_queue.cc:229] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c563af166844998a91fd07b50277c01" member_type: VOTER } I0609 06:56:48.080891 1479 raft_consensus.cc:380] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:48.081094 1479 raft_consensus.cc:472] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:48.081351 1479 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:48.084975 1479 raft_consensus.cc:494] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c563af166844998a91fd07b50277c01" member_type: VOTER } I0609 06:56:48.085435 1479 leader_election.cc:310] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c563af166844998a91fd07b50277c01; no voters: I0609 06:56:48.089617 1479 leader_election.cc:296] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:48.089942 1481 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:48.094367 1481 raft_consensus.cc:667] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [term 1 LEADER]: Becoming Leader. State: Replica: 9c563af166844998a91fd07b50277c01, State: Running, Role: LEADER I0609 06:56:48.095059 1481 consensus_queue.cc:206] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c563af166844998a91fd07b50277c01" member_type: VOTER } I0609 06:56:48.098661 1482 sys_catalog.cc:338] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c563af166844998a91fd07b50277c01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c563af166844998a91fd07b50277c01" member_type: VOTER } } I0609 06:56:48.099025 1483 sys_catalog.cc:338] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c563af166844998a91fd07b50277c01. Latest consensus state: current_term: 1 leader_uuid: "9c563af166844998a91fd07b50277c01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c563af166844998a91fd07b50277c01" member_type: VOTER } } I0609 06:56:48.099434 1482 sys_catalog.cc:341] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [sys.catalog]: This master's current role is: LEADER I0609 06:56:48.099611 1483 sys_catalog.cc:341] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [sys.catalog]: This master's current role is: LEADER I0609 06:56:48.100021 1479 sys_catalog.cc:338] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "9c563af166844998a91fd07b50277c01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c563af166844998a91fd07b50277c01" member_type: VOTER } } I0609 06:56:48.100822 1479 sys_catalog.cc:341] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [sys.catalog]: This master's current role is: LEADER I0609 06:56:48.102144 1479 sys_catalog.cc:426] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:48.102659 1485 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:48.105872 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:48.107641 1485 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:48.140517 1485 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:48.142056 1485 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:48.150959 1485 catalog_manager.cc:4293] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01: Generated new TSK 0 I0609 06:56:48.184751 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 91016us I0609 06:56:48.185103 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:48.185235 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:48.185443 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:48.185555 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:48.186404 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "1ef76c1b102746218f66b3fde5ddbc37" format_stamp: "Formatted at 2019-06-09 06:56:48 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:48.195291 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.009s user 0.007s sys 0.002s I0609 06:56:48.196647 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:48.203930 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:48.204080 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/ts-0-root uuid: "1ef76c1b102746218f66b3fde5ddbc37" format_stamp: "Formatted at 2019-06-09 06:56:48 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:48.204286 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:48.236112 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:48.236250 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:48.279413 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:39929 I0609 06:56:48.281695 1119 webserver.cc:374] Webserver started at http://127.1.23.193:42205/ using document root and password file I0609 06:56:48.293728 1553 heartbeater.cc:324] Connected to a master server at 127.1.23.254:40485 I0609 06:56:48.293990 1553 heartbeater.cc:405] Registering TS with master... I0609 06:56:48.294389 1553 heartbeater.cc:454] Master 127.1.23.254:40485 requested a full tablet report, sending... I0609 06:56:48.295825 1446 ts_manager.cc:161] Registered new tserver with Master: 1ef76c1b102746218f66b3fde5ddbc37 (127.1.23.193:39929) I0609 06:56:48.296869 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.013678975s I0609 06:56:48.297682 1446 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36308 I0609 06:56:48.310493 1553 heartbeater.cc:446] Master 127.1.23.254:40485 was elected leader, sending a full tablet report... I0609 06:56:48.314528 1445 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36310: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:48.340188 1520 tablet_service.cc:1016] Processing CreateTablet for tablet bec08078408245a9be6e31bd01edb05c (table=client-testtb [id=3a0043a94fda45c897754fc1c2cd78bb]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:48.340430 1519 tablet_service.cc:1016] Processing CreateTablet for tablet e3536a1a17a04767ada7f39efb2c2f3a (table=client-testtb [id=3a0043a94fda45c897754fc1c2cd78bb]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:48.340936 1520 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet bec08078408245a9be6e31bd01edb05c. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:48.341475 1519 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet e3536a1a17a04767ada7f39efb2c2f3a. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:48.352306 1565 tablet_bootstrap.cc:439] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37: Bootstrap starting. I0609 06:56:48.354651 1565 tablet_bootstrap.cc:590] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37: No blocks or log segments found. Creating new log. I0609 06:56:48.357465 1565 tablet_bootstrap.cc:439] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37: No bootstrap required, opened a new log I0609 06:56:48.357761 1565 ts_tablet_manager.cc:1054] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s I0609 06:56:48.358858 1565 raft_consensus.cc:340] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.359220 1565 raft_consensus.cc:366] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:48.359401 1565 raft_consensus.cc:705] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef76c1b102746218f66b3fde5ddbc37, State: Initialized, Role: FOLLOWER I0609 06:56:48.359828 1565 consensus_queue.cc:229] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.360229 1565 raft_consensus.cc:380] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:48.360460 1565 raft_consensus.cc:472] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:48.360734 1565 raft_consensus.cc:2890] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:48.364395 1565 raft_consensus.cc:494] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.364863 1565 leader_election.cc:310] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef76c1b102746218f66b3fde5ddbc37; no voters: I0609 06:56:48.366165 1565 leader_election.cc:296] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:48.366375 1567 raft_consensus.cc:2644] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:48.367924 1567 raft_consensus.cc:667] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 LEADER]: Becoming Leader. State: Replica: 1ef76c1b102746218f66b3fde5ddbc37, State: Running, Role: LEADER I0609 06:56:48.369910 1565 ts_tablet_manager.cc:1082] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37: Time spent starting tablet: real 0.012s user 0.006s sys 0.004s I0609 06:56:48.370368 1565 tablet_bootstrap.cc:439] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37: Bootstrap starting. I0609 06:56:48.372129 1565 tablet_bootstrap.cc:590] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37: No blocks or log segments found. Creating new log. I0609 06:56:48.368979 1567 consensus_queue.cc:206] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.375944 1565 tablet_bootstrap.cc:439] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37: No bootstrap required, opened a new log I0609 06:56:48.376308 1565 ts_tablet_manager.cc:1054] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s I0609 06:56:48.378365 1565 raft_consensus.cc:340] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.378818 1565 raft_consensus.cc:366] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:48.378999 1565 raft_consensus.cc:705] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef76c1b102746218f66b3fde5ddbc37, State: Initialized, Role: FOLLOWER I0609 06:56:48.379019 1445 catalog_manager.cc:3989] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 reported cstate change: term changed from 0 to 1, leader changed from to 1ef76c1b102746218f66b3fde5ddbc37 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "1ef76c1b102746218f66b3fde5ddbc37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } health_report { overall_health: HEALTHY } } } I0609 06:56:48.379453 1565 consensus_queue.cc:229] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.379982 1565 raft_consensus.cc:380] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:48.380198 1565 raft_consensus.cc:472] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:48.380427 1565 raft_consensus.cc:2890] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:48.384519 1565 raft_consensus.cc:494] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.384961 1565 leader_election.cc:310] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef76c1b102746218f66b3fde5ddbc37; no voters: I0609 06:56:48.385592 1565 leader_election.cc:296] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:48.385704 1567 raft_consensus.cc:2644] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:48.386121 1567 raft_consensus.cc:667] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 LEADER]: Becoming Leader. State: Replica: 1ef76c1b102746218f66b3fde5ddbc37, State: Running, Role: LEADER I0609 06:56:48.386595 1565 ts_tablet_manager.cc:1082] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37: Time spent starting tablet: real 0.010s user 0.006s sys 0.003s I0609 06:56:48.386552 1567 consensus_queue.cc:206] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.391312 1445 catalog_manager.cc:3989] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 reported cstate change: term changed from 0 to 1, leader changed from to 1ef76c1b102746218f66b3fde5ddbc37 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "1ef76c1b102746218f66b3fde5ddbc37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } health_report { overall_health: HEALTHY } } } I0609 06:56:48.410967 1445 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36310: name: "TestNonFaultTolerantScannerExpired" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:48.422700 1520 tablet_service.cc:1016] Processing CreateTablet for tablet c6181f8c492242c49c55a418cdca32da (table=TestNonFaultTolerantScannerExpired [id=7bbe86093172406a9e47f6b33bdbfa13]), partition=RANGE (key) PARTITION UNBOUNDED I0609 06:56:48.423274 1520 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet c6181f8c492242c49c55a418cdca32da. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:48.430996 1565 tablet_bootstrap.cc:439] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37: Bootstrap starting. I0609 06:56:48.433679 1565 tablet_bootstrap.cc:590] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37: No blocks or log segments found. Creating new log. I0609 06:56:48.436522 1565 tablet_bootstrap.cc:439] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37: No bootstrap required, opened a new log I0609 06:56:48.436805 1565 ts_tablet_manager.cc:1054] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s I0609 06:56:48.438175 1565 raft_consensus.cc:340] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.438545 1565 raft_consensus.cc:366] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:48.438748 1565 raft_consensus.cc:705] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef76c1b102746218f66b3fde5ddbc37, State: Initialized, Role: FOLLOWER I0609 06:56:48.439254 1565 consensus_queue.cc:229] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.439738 1565 raft_consensus.cc:380] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:48.439924 1565 raft_consensus.cc:472] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:48.440155 1565 raft_consensus.cc:2890] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:48.443336 1565 raft_consensus.cc:494] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.443939 1565 leader_election.cc:310] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef76c1b102746218f66b3fde5ddbc37; no voters: I0609 06:56:48.444442 1565 leader_election.cc:296] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:48.444576 1567 raft_consensus.cc:2644] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:48.444974 1567 raft_consensus.cc:667] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 LEADER]: Becoming Leader. State: Replica: 1ef76c1b102746218f66b3fde5ddbc37, State: Running, Role: LEADER I0609 06:56:48.445529 1565 ts_tablet_manager.cc:1082] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s I0609 06:56:48.445619 1567 consensus_queue.cc:206] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.449496 1446 catalog_manager.cc:3989] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 reported cstate change: term changed from 0 to 1, leader changed from to 1ef76c1b102746218f66b3fde5ddbc37 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "1ef76c1b102746218f66b3fde5ddbc37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } health_report { overall_health: HEALTHY } } } I0609 06:56:48.804638 1119 client-test.cc:652] Restarting TS at 127.1.23.193:39929 I0609 06:56:48.805017 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:48.831907 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:48.833500 1119 tablet_replica.cc:265] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37: stopping tablet replica I0609 06:56:48.833863 1119 raft_consensus.cc:2147] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:48.834110 1119 raft_consensus.cc:2174] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:48.836693 1119 tablet_replica.cc:265] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37: stopping tablet replica I0609 06:56:48.836987 1119 raft_consensus.cc:2147] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:48.837226 1119 raft_consensus.cc:2174] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:48.839992 1119 tablet_replica.cc:265] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37: stopping tablet replica I0609 06:56:48.840349 1119 raft_consensus.cc:2147] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:48.840593 1119 raft_consensus.cc:2174] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:48.854550 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:48.854676 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:48.854960 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:48.875669 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 91016us I0609 06:56:48.876051 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:48.876180 1119 fs_manager.cc:266] Using existing metadata directory in first data directory I0609 06:56:48.878844 1119 fs_manager.cc:399] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s I0609 06:56:48.880012 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:48.886605 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:48.886745 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/ts-0-root uuid: "1ef76c1b102746218f66b3fde5ddbc37" format_stamp: "Formatted at 2019-06-09 06:56:48 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:48.886983 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestNonFaultTolerantScannerExpired.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:48.914347 1119 ts_tablet_manager.cc:344] Loading tablet metadata (0/3 complete) I0609 06:56:48.924500 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (3 total tablets, 3 live tablets) I0609 06:56:48.924659 1119 ts_tablet_manager.cc:362] Registering tablets (0/3 complete) I0609 06:56:48.929967 1589 tablet_bootstrap.cc:439] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37: Bootstrap starting. I0609 06:56:48.935808 1119 ts_tablet_manager.cc:376] Registered 3 tablets I0609 06:56:48.941601 1589 tablet_bootstrap.cc:439] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates I0609 06:56:48.942756 1589 tablet_bootstrap.cc:439] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37: Bootstrap complete. I0609 06:56:48.943329 1589 ts_tablet_manager.cc:1054] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s I0609 06:56:48.944985 1589 raft_consensus.cc:340] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.945595 1589 raft_consensus.cc:705] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef76c1b102746218f66b3fde5ddbc37, State: Initialized, Role: FOLLOWER I0609 06:56:48.946264 1589 consensus_queue.cc:229] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.946887 1589 raft_consensus.cc:380] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:48.947144 1589 raft_consensus.cc:472] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:48.947475 1589 raft_consensus.cc:2890] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Advancing to term 2 I0609 06:56:48.953316 1589 raft_consensus.cc:494] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.954378 1589 leader_election.cc:310] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef76c1b102746218f66b3fde5ddbc37; no voters: I0609 06:56:48.958493 1589 leader_election.cc:296] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [CANDIDATE]: Term 2 election: Requested vote from peers I0609 06:56:48.958678 1598 raft_consensus.cc:2644] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 FOLLOWER]: Leader election won for term 2 I0609 06:56:48.961500 1598 raft_consensus.cc:667] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 LEADER]: Becoming Leader. State: Replica: 1ef76c1b102746218f66b3fde5ddbc37, State: Running, Role: LEADER I0609 06:56:48.962479 1589 ts_tablet_manager.cc:1082] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37: Time spent starting tablet: real 0.019s user 0.013s sys 0.005s I0609 06:56:48.963390 1589 tablet_bootstrap.cc:439] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37: Bootstrap starting. I0609 06:56:48.962832 1598 consensus_queue.cc:206] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.976982 1589 tablet_bootstrap.cc:439] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates I0609 06:56:48.978101 1589 tablet_bootstrap.cc:439] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37: Bootstrap complete. I0609 06:56:48.978674 1589 ts_tablet_manager.cc:1054] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s I0609 06:56:48.980515 1589 raft_consensus.cc:340] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.981159 1589 raft_consensus.cc:705] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef76c1b102746218f66b3fde5ddbc37, State: Initialized, Role: FOLLOWER I0609 06:56:48.981994 1589 consensus_queue.cc:229] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.982674 1589 raft_consensus.cc:380] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:48.982949 1589 raft_consensus.cc:472] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:48.983347 1589 raft_consensus.cc:2890] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Advancing to term 2 I0609 06:56:48.989645 1589 raft_consensus.cc:494] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.990290 1589 leader_election.cc:310] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef76c1b102746218f66b3fde5ddbc37; no voters: I0609 06:56:48.990949 1589 leader_election.cc:296] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [CANDIDATE]: Term 2 election: Requested vote from peers I0609 06:56:48.991091 1598 raft_consensus.cc:2644] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 FOLLOWER]: Leader election won for term 2 I0609 06:56:48.991641 1598 raft_consensus.cc:667] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 LEADER]: Becoming Leader. State: Replica: 1ef76c1b102746218f66b3fde5ddbc37, State: Running, Role: LEADER I0609 06:56:48.992252 1589 ts_tablet_manager.cc:1082] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s I0609 06:56:48.992197 1598 consensus_queue.cc:206] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:48.992813 1589 tablet_bootstrap.cc:439] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37: Bootstrap starting. I0609 06:56:49.061743 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:39929 I0609 06:56:49.065729 1119 webserver.cc:374] Webserver started at http://127.1.23.193:42205/ using document root and password file I0609 06:56:49.070906 1119 client-test.cc:656] Waiting for TS 127.1.23.193:39929 to finish bootstrapping I0609 06:56:49.093118 1653 heartbeater.cc:324] Connected to a master server at 127.1.23.254:40485 I0609 06:56:49.093456 1653 heartbeater.cc:405] Registering TS with master... I0609 06:56:49.093973 1653 heartbeater.cc:454] Master 127.1.23.254:40485 requested a full tablet report, sending... I0609 06:56:49.096709 1444 ts_manager.cc:161] Re-registered known tserver with Master: 1ef76c1b102746218f66b3fde5ddbc37 (127.1.23.193:39929) I0609 06:56:49.098078 1444 catalog_manager.cc:3989] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "1ef76c1b102746218f66b3fde5ddbc37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } health_report { overall_health: HEALTHY } } } I0609 06:56:49.099118 1444 catalog_manager.cc:3989] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "1ef76c1b102746218f66b3fde5ddbc37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } health_report { overall_health: HEALTHY } } } I0609 06:56:49.117342 1444 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36316 I0609 06:56:49.130353 1653 heartbeater.cc:446] Master 127.1.23.254:40485 was elected leader, sending a full tablet report... I0609 06:56:49.228212 1589 tablet_bootstrap.cc:439] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1000 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates I0609 06:56:49.228910 1589 tablet_bootstrap.cc:439] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37: Bootstrap complete. I0609 06:56:49.229363 1589 ts_tablet_manager.cc:1054] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37: Time spent bootstrapping tablet: real 0.237s user 0.207s sys 0.014s I0609 06:56:49.230435 1589 raft_consensus.cc:340] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:49.230801 1589 raft_consensus.cc:705] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef76c1b102746218f66b3fde5ddbc37, State: Initialized, Role: FOLLOWER I0609 06:56:49.231281 1589 consensus_queue.cc:229] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:49.231693 1589 raft_consensus.cc:380] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:49.231873 1589 raft_consensus.cc:472] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:49.232113 1589 raft_consensus.cc:2890] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 1 FOLLOWER]: Advancing to term 2 I0609 06:56:49.235994 1589 raft_consensus.cc:494] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:49.236390 1589 leader_election.cc:310] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef76c1b102746218f66b3fde5ddbc37; no voters: I0609 06:56:49.236796 1589 leader_election.cc:296] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [CANDIDATE]: Term 2 election: Requested vote from peers I0609 06:56:49.236917 1598 raft_consensus.cc:2644] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 FOLLOWER]: Leader election won for term 2 I0609 06:56:49.237381 1598 raft_consensus.cc:667] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 LEADER]: Becoming Leader. State: Replica: 1ef76c1b102746218f66b3fde5ddbc37, State: Running, Role: LEADER I0609 06:56:49.237924 1589 ts_tablet_manager.cc:1082] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s I0609 06:56:49.237854 1598 consensus_queue.cc:206] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } } I0609 06:56:49.248103 1444 catalog_manager.cc:3989] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "1ef76c1b102746218f66b3fde5ddbc37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76c1b102746218f66b3fde5ddbc37" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 39929 } health_report { overall_health: HEALTHY } } } I0609 06:56:49.259812 1606 tablet_service.cc:2505] Scan: Not found: Scanner 1d860289b41c4a659d5d2b89551faa08 not found (it may have expired): call sequence id=1, remote={username='slave'} at 127.0.0.1:56630 I0609 06:56:49.263203 1119 tablet_server.cc:155] TabletServer shutting down... W0609 06:56:49.263262 1560 client.cc:1440] Couldn't close scanner 1d860289b41c4a659d5d2b89551faa08: Service unavailable: reactor is shutting down (error 108) W0609 06:56:49.271795 1584 connection.cc:647] server connection from 127.0.0.1:56630 recv error: Network error: recv error from 0.0.0.0:0: Transport endpoint is not connected (error 107) I0609 06:56:49.295591 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:49.296094 1119 tablet_replica.cc:265] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37: stopping tablet replica I0609 06:56:49.296577 1119 raft_consensus.cc:2147] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 LEADER]: Raft consensus shutting down. I0609 06:56:49.296937 1119 raft_consensus.cc:2174] T c6181f8c492242c49c55a418cdca32da P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 FOLLOWER]: Raft consensus is shut down! I0609 06:56:49.301666 1119 tablet_replica.cc:265] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37: stopping tablet replica I0609 06:56:49.302084 1119 raft_consensus.cc:2147] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 LEADER]: Raft consensus shutting down. I0609 06:56:49.302433 1119 raft_consensus.cc:2174] T bec08078408245a9be6e31bd01edb05c P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 FOLLOWER]: Raft consensus is shut down! I0609 06:56:49.305217 1119 tablet_replica.cc:265] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37: stopping tablet replica I0609 06:56:49.305637 1119 raft_consensus.cc:2147] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 LEADER]: Raft consensus shutting down. I0609 06:56:49.305961 1119 raft_consensus.cc:2174] T e3536a1a17a04767ada7f39efb2c2f3a P 1ef76c1b102746218f66b3fde5ddbc37 [term 2 FOLLOWER]: Raft consensus is shut down! I0609 06:56:49.321462 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:49.321607 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:49.321919 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:49.329254 1119 master.cc:247] Master@127.1.23.254:40485 shutting down... I0609 06:56:49.346165 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:49.346427 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:49.346690 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 9c563af166844998a91fd07b50277c01: stopping tablet replica I0609 06:56:49.356722 1119 master.cc:258] Master@127.1.23.254:40485 shutdown complete. [ OK ] ClientTest.TestNonFaultTolerantScannerExpired (1560 ms) [ RUN ] ClientTest.TestMinMaxRangeBounds I0609 06:56:49.370496 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:45405 I0609 06:56:49.382620 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 91516us I0609 06:56:49.382946 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:49.383093 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:49.383304 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:49.383428 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:49.384299 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMinMaxRangeBounds.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "58538d5211d24f628c7c3bdce9bb8e0c" format_stamp: "Formatted at 2019-06-09 06:56:49 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:49.392294 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.010s sys 0.000s I0609 06:56:49.393642 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:49.400096 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s I0609 06:56:49.400252 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMinMaxRangeBounds.1560063404629499-1119/minicluster-data/master-0-root uuid: "58538d5211d24f628c7c3bdce9bb8e0c" format_stamp: "Formatted at 2019-06-09 06:56:49 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:49.400473 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMinMaxRangeBounds.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMinMaxRangeBounds.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMinMaxRangeBounds.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:49.625679 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:45405 I0609 06:56:49.627688 1119 webserver.cc:374] Webserver started at http://127.1.23.254:41305/ using document root and password file I0609 06:56:49.629714 1717 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:49.636785 1717 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c: Bootstrap starting. I0609 06:56:49.638833 1717 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c: No blocks or log segments found. Creating new log. I0609 06:56:49.642426 1717 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c: No bootstrap required, opened a new log I0609 06:56:49.643599 1717 raft_consensus.cc:340] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58538d5211d24f628c7c3bdce9bb8e0c" member_type: VOTER } I0609 06:56:49.643908 1717 raft_consensus.cc:366] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:49.644129 1717 raft_consensus.cc:705] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58538d5211d24f628c7c3bdce9bb8e0c, State: Initialized, Role: FOLLOWER I0609 06:56:49.644560 1717 consensus_queue.cc:229] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58538d5211d24f628c7c3bdce9bb8e0c" member_type: VOTER } I0609 06:56:49.644929 1717 raft_consensus.cc:380] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:49.645112 1717 raft_consensus.cc:472] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:49.645395 1717 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:49.648285 1717 raft_consensus.cc:494] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58538d5211d24f628c7c3bdce9bb8e0c" member_type: VOTER } I0609 06:56:49.648670 1717 leader_election.cc:310] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58538d5211d24f628c7c3bdce9bb8e0c; no voters: I0609 06:56:49.653602 1717 leader_election.cc:296] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:49.653759 1719 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:49.655318 1719 raft_consensus.cc:667] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [term 1 LEADER]: Becoming Leader. State: Replica: 58538d5211d24f628c7c3bdce9bb8e0c, State: Running, Role: LEADER I0609 06:56:49.655776 1719 consensus_queue.cc:206] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58538d5211d24f628c7c3bdce9bb8e0c" member_type: VOTER } I0609 06:56:49.657954 1720 sys_catalog.cc:338] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58538d5211d24f628c7c3bdce9bb8e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58538d5211d24f628c7c3bdce9bb8e0c" member_type: VOTER } } I0609 06:56:49.658535 1720 sys_catalog.cc:341] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [sys.catalog]: This master's current role is: LEADER I0609 06:56:49.658588 1717 sys_catalog.cc:338] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "58538d5211d24f628c7c3bdce9bb8e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58538d5211d24f628c7c3bdce9bb8e0c" member_type: VOTER } } I0609 06:56:49.658979 1721 sys_catalog.cc:338] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58538d5211d24f628c7c3bdce9bb8e0c. Latest consensus state: current_term: 1 leader_uuid: "58538d5211d24f628c7c3bdce9bb8e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58538d5211d24f628c7c3bdce9bb8e0c" member_type: VOTER } } I0609 06:56:49.659289 1717 sys_catalog.cc:341] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [sys.catalog]: This master's current role is: LEADER I0609 06:56:49.659484 1721 sys_catalog.cc:341] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [sys.catalog]: This master's current role is: LEADER I0609 06:56:49.666432 1723 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:49.668587 1723 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:49.673449 1717 sys_catalog.cc:426] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:49.677753 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:49.703024 1723 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:49.704509 1723 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:49.712607 1723 catalog_manager.cc:4293] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c: Generated new TSK 0 I0609 06:56:49.756155 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 91516us I0609 06:56:49.756649 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:49.756862 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:49.757144 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:49.757333 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:49.758466 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMinMaxRangeBounds.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "bad01a6e9aa649e18994bfa51c1bbd94" format_stamp: "Formatted at 2019-06-09 06:56:49 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:49.768345 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.009s user 0.006s sys 0.004s I0609 06:56:49.769693 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:49.777029 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:49.777184 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMinMaxRangeBounds.1560063404629499-1119/minicluster-data/ts-0-root uuid: "bad01a6e9aa649e18994bfa51c1bbd94" format_stamp: "Formatted at 2019-06-09 06:56:49 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:49.777472 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMinMaxRangeBounds.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMinMaxRangeBounds.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMinMaxRangeBounds.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:49.832191 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:49.832365 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:50.009068 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:42979 I0609 06:56:50.021728 1119 webserver.cc:374] Webserver started at http://127.1.23.193:35713/ using document root and password file I0609 06:56:50.038425 1791 heartbeater.cc:324] Connected to a master server at 127.1.23.254:45405 I0609 06:56:50.038659 1791 heartbeater.cc:405] Registering TS with master... I0609 06:56:50.039047 1791 heartbeater.cc:454] Master 127.1.23.254:45405 requested a full tablet report, sending... I0609 06:56:50.040565 1684 ts_manager.cc:161] Registered new tserver with Master: bad01a6e9aa649e18994bfa51c1bbd94 (127.1.23.193:42979) I0609 06:56:50.041231 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.003624063s I0609 06:56:50.042302 1684 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60770 I0609 06:56:50.054445 1791 heartbeater.cc:446] Master 127.1.23.254:45405 was elected leader, sending a full tablet report... I0609 06:56:50.081779 1684 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60772: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:50.110759 1758 tablet_service.cc:1016] Processing CreateTablet for tablet 759a30de014344b0b003240a7a906123 (table=client-testtb [id=062d103cfab049c795c7a2303c7b2fba]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:50.111515 1758 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 759a30de014344b0b003240a7a906123. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:50.111243 1757 tablet_service.cc:1016] Processing CreateTablet for tablet 299b3859605e459a92cf0cb49e452f33 (table=client-testtb [id=062d103cfab049c795c7a2303c7b2fba]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:50.112156 1757 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 299b3859605e459a92cf0cb49e452f33. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:50.126034 1803 tablet_bootstrap.cc:439] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94: Bootstrap starting. I0609 06:56:50.128557 1803 tablet_bootstrap.cc:590] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94: No blocks or log segments found. Creating new log. I0609 06:56:50.138521 1803 tablet_bootstrap.cc:439] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94: No bootstrap required, opened a new log I0609 06:56:50.138905 1803 ts_tablet_manager.cc:1054] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s I0609 06:56:50.140678 1803 raft_consensus.cc:340] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } } I0609 06:56:50.141250 1803 raft_consensus.cc:366] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:50.141525 1803 raft_consensus.cc:705] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad01a6e9aa649e18994bfa51c1bbd94, State: Initialized, Role: FOLLOWER I0609 06:56:50.142153 1803 consensus_queue.cc:229] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } } I0609 06:56:50.143157 1803 raft_consensus.cc:380] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:50.143388 1803 raft_consensus.cc:472] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:50.143661 1803 raft_consensus.cc:2890] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:50.147538 1803 raft_consensus.cc:494] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } } I0609 06:56:50.148053 1803 leader_election.cc:310] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bad01a6e9aa649e18994bfa51c1bbd94; no voters: I0609 06:56:50.149605 1803 leader_election.cc:296] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:50.149878 1805 raft_consensus.cc:2644] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:50.151707 1803 ts_tablet_manager.cc:1082] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s I0609 06:56:50.152127 1803 tablet_bootstrap.cc:439] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94: Bootstrap starting. I0609 06:56:50.153944 1805 raft_consensus.cc:667] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 LEADER]: Becoming Leader. State: Replica: bad01a6e9aa649e18994bfa51c1bbd94, State: Running, Role: LEADER I0609 06:56:50.154170 1803 tablet_bootstrap.cc:590] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94: No blocks or log segments found. Creating new log. I0609 06:56:50.154522 1805 consensus_queue.cc:206] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } } I0609 06:56:50.159826 1803 tablet_bootstrap.cc:439] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94: No bootstrap required, opened a new log I0609 06:56:50.160255 1803 ts_tablet_manager.cc:1054] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s I0609 06:56:50.160264 1684 catalog_manager.cc:3989] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 reported cstate change: term changed from 0 to 1, leader changed from to bad01a6e9aa649e18994bfa51c1bbd94 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } health_report { overall_health: HEALTHY } } } I0609 06:56:50.163561 1803 raft_consensus.cc:340] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } } I0609 06:56:50.164072 1803 raft_consensus.cc:366] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:50.164337 1803 raft_consensus.cc:705] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad01a6e9aa649e18994bfa51c1bbd94, State: Initialized, Role: FOLLOWER I0609 06:56:50.165087 1803 consensus_queue.cc:229] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } } I0609 06:56:50.165750 1803 raft_consensus.cc:380] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:50.166028 1803 raft_consensus.cc:472] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:50.166378 1803 raft_consensus.cc:2890] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:50.172260 1803 raft_consensus.cc:494] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } } I0609 06:56:50.172694 1803 leader_election.cc:310] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bad01a6e9aa649e18994bfa51c1bbd94; no voters: I0609 06:56:50.173154 1803 leader_election.cc:296] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:50.173347 1807 raft_consensus.cc:2644] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:50.173835 1807 raft_consensus.cc:667] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 LEADER]: Becoming Leader. State: Replica: bad01a6e9aa649e18994bfa51c1bbd94, State: Running, Role: LEADER I0609 06:56:50.174353 1803 ts_tablet_manager.cc:1082] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94: Time spent starting tablet: real 0.014s user 0.011s sys 0.000s I0609 06:56:50.174523 1807 consensus_queue.cc:206] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } } I0609 06:56:50.178458 1684 catalog_manager.cc:3989] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 reported cstate change: term changed from 0 to 1, leader changed from to bad01a6e9aa649e18994bfa51c1bbd94 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } health_report { overall_health: HEALTHY } } } I0609 06:56:50.193418 1684 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60772: name: "TestMinMaxRangeBounds" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "value" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\010\001\000\000\000\200\t\001\377\377\377\177" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:50.207790 1758 tablet_service.cc:1016] Processing CreateTablet for tablet 112649f2817345279bb899b96a6c26d1 (table=TestMinMaxRangeBounds [id=0defa11903f040a1824925f5e55ad909]), partition=RANGE (key) PARTITION VALUES >= -2147483647 I0609 06:56:50.208719 1758 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 112649f2817345279bb899b96a6c26d1. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:50.220219 1803 tablet_bootstrap.cc:439] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94: Bootstrap starting. I0609 06:56:50.222360 1803 tablet_bootstrap.cc:590] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94: No blocks or log segments found. Creating new log. I0609 06:56:50.226157 1803 tablet_bootstrap.cc:439] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94: No bootstrap required, opened a new log I0609 06:56:50.226469 1803 ts_tablet_manager.cc:1054] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s I0609 06:56:50.227664 1803 raft_consensus.cc:340] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } } I0609 06:56:50.228057 1803 raft_consensus.cc:366] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:50.228226 1803 raft_consensus.cc:705] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad01a6e9aa649e18994bfa51c1bbd94, State: Initialized, Role: FOLLOWER I0609 06:56:50.228706 1803 consensus_queue.cc:229] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } } I0609 06:56:50.229127 1803 raft_consensus.cc:380] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:50.229395 1803 raft_consensus.cc:472] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:50.229655 1803 raft_consensus.cc:2890] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:50.233022 1803 raft_consensus.cc:494] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } } I0609 06:56:50.233466 1803 leader_election.cc:310] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bad01a6e9aa649e18994bfa51c1bbd94; no voters: I0609 06:56:50.233909 1803 leader_election.cc:296] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:50.234051 1807 raft_consensus.cc:2644] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:50.234462 1807 raft_consensus.cc:667] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 LEADER]: Becoming Leader. State: Replica: bad01a6e9aa649e18994bfa51c1bbd94, State: Running, Role: LEADER I0609 06:56:50.235110 1803 ts_tablet_manager.cc:1082] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s I0609 06:56:50.234946 1807 consensus_queue.cc:206] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } } I0609 06:56:50.239105 1683 catalog_manager.cc:3989] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 reported cstate change: term changed from 0 to 1, leader changed from to bad01a6e9aa649e18994bfa51c1bbd94 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bad01a6e9aa649e18994bfa51c1bbd94" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 42979 } health_report { overall_health: HEALTHY } } } I0609 06:56:50.248864 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:50.273018 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:50.273586 1119 tablet_replica.cc:265] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94: stopping tablet replica I0609 06:56:50.274085 1119 raft_consensus.cc:2147] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:50.274426 1119 raft_consensus.cc:2174] T 112649f2817345279bb899b96a6c26d1 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:50.277371 1119 tablet_replica.cc:265] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94: stopping tablet replica I0609 06:56:50.277849 1119 raft_consensus.cc:2147] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:50.278188 1119 raft_consensus.cc:2174] T 299b3859605e459a92cf0cb49e452f33 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:50.281018 1119 tablet_replica.cc:265] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94: stopping tablet replica I0609 06:56:50.281453 1119 raft_consensus.cc:2147] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:50.281770 1119 raft_consensus.cc:2174] T 759a30de014344b0b003240a7a906123 P bad01a6e9aa649e18994bfa51c1bbd94 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:50.296690 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:50.296845 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:50.297122 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:50.303210 1119 master.cc:247] Master@127.1.23.254:45405 shutting down... I0609 06:56:50.320876 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:50.321133 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:50.321382 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 58538d5211d24f628c7c3bdce9bb8e0c: stopping tablet replica I0609 06:56:50.332306 1119 master.cc:258] Master@127.1.23.254:45405 shutdown complete. [ OK ] ClientTest.TestMinMaxRangeBounds (975 ms) [ RUN ] ClientTest.TestInsertAutoFlushSync I0609 06:56:50.345141 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:42461 I0609 06:56:50.357235 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 92016us I0609 06:56:50.357544 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:50.357712 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:50.357901 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:50.358022 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:50.358942 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestInsertAutoFlushSync.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "2f1d15ef884540a39de34977b2c2b45f" format_stamp: "Formatted at 2019-06-09 06:56:50 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:50.367471 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s I0609 06:56:50.368970 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:50.386108 1119 fs_manager.cc:419] Time spent opening block manager: real 0.011s user 0.004s sys 0.007s I0609 06:56:50.386255 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestInsertAutoFlushSync.1560063404629499-1119/minicluster-data/master-0-root uuid: "2f1d15ef884540a39de34977b2c2b45f" format_stamp: "Formatted at 2019-06-09 06:56:50 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:50.386472 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestInsertAutoFlushSync.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestInsertAutoFlushSync.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestInsertAutoFlushSync.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:50.521406 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:42461 I0609 06:56:50.523806 1119 webserver.cc:374] Webserver started at http://127.1.23.254:45857/ using document root and password file I0609 06:56:50.526343 1868 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:50.541402 1868 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f: Bootstrap starting. I0609 06:56:50.543395 1868 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f: No blocks or log segments found. Creating new log. I0609 06:56:50.554538 1868 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f: No bootstrap required, opened a new log I0609 06:56:50.557067 1868 raft_consensus.cc:340] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1d15ef884540a39de34977b2c2b45f" member_type: VOTER } I0609 06:56:50.557598 1868 raft_consensus.cc:366] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:50.557902 1868 raft_consensus.cc:705] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f1d15ef884540a39de34977b2c2b45f, State: Initialized, Role: FOLLOWER I0609 06:56:50.558490 1868 consensus_queue.cc:229] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1d15ef884540a39de34977b2c2b45f" member_type: VOTER } I0609 06:56:50.559098 1868 raft_consensus.cc:380] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:50.559353 1868 raft_consensus.cc:472] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:50.559669 1868 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:50.564435 1868 raft_consensus.cc:494] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1d15ef884540a39de34977b2c2b45f" member_type: VOTER } I0609 06:56:50.564929 1868 leader_election.cc:310] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f1d15ef884540a39de34977b2c2b45f; no voters: I0609 06:56:50.566412 1868 leader_election.cc:296] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:50.566578 1870 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:50.569566 1870 raft_consensus.cc:667] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [term 1 LEADER]: Becoming Leader. State: Replica: 2f1d15ef884540a39de34977b2c2b45f, State: Running, Role: LEADER I0609 06:56:50.569964 1870 consensus_queue.cc:206] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1d15ef884540a39de34977b2c2b45f" member_type: VOTER } I0609 06:56:50.574563 1871 sys_catalog.cc:338] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f1d15ef884540a39de34977b2c2b45f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1d15ef884540a39de34977b2c2b45f" member_type: VOTER } } I0609 06:56:50.574625 1872 sys_catalog.cc:338] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f1d15ef884540a39de34977b2c2b45f. Latest consensus state: current_term: 1 leader_uuid: "2f1d15ef884540a39de34977b2c2b45f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1d15ef884540a39de34977b2c2b45f" member_type: VOTER } } I0609 06:56:50.575160 1872 sys_catalog.cc:341] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [sys.catalog]: This master's current role is: LEADER I0609 06:56:50.575181 1871 sys_catalog.cc:341] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [sys.catalog]: This master's current role is: LEADER I0609 06:56:50.575785 1868 sys_catalog.cc:338] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "2f1d15ef884540a39de34977b2c2b45f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1d15ef884540a39de34977b2c2b45f" member_type: VOTER } } I0609 06:56:50.576424 1868 sys_catalog.cc:341] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [sys.catalog]: This master's current role is: LEADER I0609 06:56:50.578480 1868 sys_catalog.cc:426] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:50.579084 1875 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:50.581120 1875 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:50.581914 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:50.611049 1875 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:50.612449 1875 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:50.623675 1875 catalog_manager.cc:4293] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f: Generated new TSK 0 I0609 06:56:50.660789 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 92016us I0609 06:56:50.661101 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:50.661242 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:50.661502 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:50.661614 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:50.662500 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestInsertAutoFlushSync.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" format_stamp: "Formatted at 2019-06-09 06:56:50 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:50.671061 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s I0609 06:56:50.672292 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:50.679592 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:50.679744 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestInsertAutoFlushSync.1560063404629499-1119/minicluster-data/ts-0-root uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" format_stamp: "Formatted at 2019-06-09 06:56:50 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:50.679966 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestInsertAutoFlushSync.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestInsertAutoFlushSync.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestInsertAutoFlushSync.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:50.696662 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:50.696830 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:50.731113 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:44979 I0609 06:56:50.732797 1119 webserver.cc:374] Webserver started at http://127.1.23.193:43321/ using document root and password file I0609 06:56:50.743718 1942 heartbeater.cc:324] Connected to a master server at 127.1.23.254:42461 I0609 06:56:50.743954 1942 heartbeater.cc:405] Registering TS with master... I0609 06:56:50.744287 1942 heartbeater.cc:454] Master 127.1.23.254:42461 requested a full tablet report, sending... I0609 06:56:50.745584 1835 ts_manager.cc:161] Registered new tserver with Master: a0f414e4d6f64a8c9af10f8dafef1faa (127.1.23.193:44979) I0609 06:56:50.745674 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.011521052s I0609 06:56:50.747248 1835 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53206 I0609 06:56:50.759434 1942 heartbeater.cc:446] Master 127.1.23.254:42461 was elected leader, sending a full tablet report... I0609 06:56:50.763259 1834 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53208: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:50.786206 1909 tablet_service.cc:1016] Processing CreateTablet for tablet 4032f27a369f49d7914e13c9354a9aa1 (table=client-testtb [id=47d76e3afb3943e5bf5ccac7ae359eb0]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:50.786610 1908 tablet_service.cc:1016] Processing CreateTablet for tablet f51c5b2f1a304bd38892d3d76596cb14 (table=client-testtb [id=47d76e3afb3943e5bf5ccac7ae359eb0]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:50.787147 1909 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 4032f27a369f49d7914e13c9354a9aa1. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:50.787614 1908 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet f51c5b2f1a304bd38892d3d76596cb14. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:50.800185 1954 tablet_bootstrap.cc:439] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa: Bootstrap starting. I0609 06:56:50.802482 1954 tablet_bootstrap.cc:590] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa: No blocks or log segments found. Creating new log. I0609 06:56:50.805083 1954 tablet_bootstrap.cc:439] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa: No bootstrap required, opened a new log I0609 06:56:50.805611 1954 ts_tablet_manager.cc:1054] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s I0609 06:56:50.806823 1954 raft_consensus.cc:340] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44979 } } I0609 06:56:50.807178 1954 raft_consensus.cc:366] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:50.807359 1954 raft_consensus.cc:705] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f414e4d6f64a8c9af10f8dafef1faa, State: Initialized, Role: FOLLOWER I0609 06:56:50.807754 1954 consensus_queue.cc:229] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44979 } } I0609 06:56:50.808189 1954 raft_consensus.cc:380] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:50.808411 1954 raft_consensus.cc:472] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:50.808678 1954 raft_consensus.cc:2890] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:50.812700 1954 raft_consensus.cc:494] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44979 } } I0609 06:56:50.813277 1954 leader_election.cc:310] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0f414e4d6f64a8c9af10f8dafef1faa; no voters: I0609 06:56:50.814504 1956 raft_consensus.cc:2644] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:50.815177 1954 leader_election.cc:296] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:50.816709 1956 raft_consensus.cc:667] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [term 1 LEADER]: Becoming Leader. State: Replica: a0f414e4d6f64a8c9af10f8dafef1faa, State: Running, Role: LEADER I0609 06:56:50.817083 1956 consensus_queue.cc:206] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44979 } } I0609 06:56:50.818578 1954 ts_tablet_manager.cc:1082] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa: Time spent starting tablet: real 0.013s user 0.009s sys 0.000s I0609 06:56:50.819003 1954 tablet_bootstrap.cc:439] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa: Bootstrap starting. I0609 06:56:50.820822 1954 tablet_bootstrap.cc:590] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa: No blocks or log segments found. Creating new log. I0609 06:56:50.824319 1834 catalog_manager.cc:3989] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa reported cstate change: term changed from 0 to 1, leader changed from to a0f414e4d6f64a8c9af10f8dafef1faa (127.1.23.193). New cstate: current_term: 1 leader_uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44979 } health_report { overall_health: HEALTHY } } } I0609 06:56:50.824961 1954 tablet_bootstrap.cc:439] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa: No bootstrap required, opened a new log I0609 06:56:50.825378 1954 ts_tablet_manager.cc:1054] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s I0609 06:56:50.827000 1954 raft_consensus.cc:340] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44979 } } I0609 06:56:50.827345 1954 raft_consensus.cc:366] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:50.827540 1954 raft_consensus.cc:705] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f414e4d6f64a8c9af10f8dafef1faa, State: Initialized, Role: FOLLOWER I0609 06:56:50.827946 1954 consensus_queue.cc:229] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44979 } } I0609 06:56:50.828519 1954 raft_consensus.cc:380] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:50.828814 1954 raft_consensus.cc:472] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:50.829094 1954 raft_consensus.cc:2890] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:50.832875 1954 raft_consensus.cc:494] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44979 } } I0609 06:56:50.833364 1954 leader_election.cc:310] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0f414e4d6f64a8c9af10f8dafef1faa; no voters: I0609 06:56:50.833971 1954 leader_election.cc:296] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:50.834076 1956 raft_consensus.cc:2644] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:50.834537 1956 raft_consensus.cc:667] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [term 1 LEADER]: Becoming Leader. State: Replica: a0f414e4d6f64a8c9af10f8dafef1faa, State: Running, Role: LEADER I0609 06:56:50.835021 1956 consensus_queue.cc:206] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44979 } } I0609 06:56:50.834997 1954 ts_tablet_manager.cc:1082] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s I0609 06:56:50.841279 1834 catalog_manager.cc:3989] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa reported cstate change: term changed from 0 to 1, leader changed from to a0f414e4d6f64a8c9af10f8dafef1faa (127.1.23.193). New cstate: current_term: 1 leader_uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f414e4d6f64a8c9af10f8dafef1faa" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 44979 } health_report { overall_health: HEALTHY } } } I0609 06:56:51.404199 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:51.423861 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:51.424113 1119 tablet_replica.cc:265] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa: stopping tablet replica I0609 06:56:51.424396 1119 raft_consensus.cc:2147] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:51.424661 1119 raft_consensus.cc:2174] T 4032f27a369f49d7914e13c9354a9aa1 P a0f414e4d6f64a8c9af10f8dafef1faa [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:51.426784 1119 tablet_replica.cc:265] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa: stopping tablet replica I0609 06:56:51.427103 1119 raft_consensus.cc:2147] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:51.427414 1119 raft_consensus.cc:2174] T f51c5b2f1a304bd38892d3d76596cb14 P a0f414e4d6f64a8c9af10f8dafef1faa [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:51.437288 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:51.437474 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:51.437767 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:51.443418 1119 master.cc:247] Master@127.1.23.254:42461 shutting down... I0609 06:56:51.457679 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:51.457953 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:51.458168 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 2f1d15ef884540a39de34977b2c2b45f: stopping tablet replica I0609 06:56:51.473799 1119 master.cc:258] Master@127.1.23.254:42461 shutdown complete. [ OK ] ClientTest.TestInsertAutoFlushSync (1141 ms) [ RUN ] ClientTest.TestWriteWithDeadTabletServer I0609 06:56:51.486852 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:46433 I0609 06:56:51.498433 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 92516us I0609 06:56:51.498757 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:51.498914 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:51.499110 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:51.499244 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:51.500068 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestWriteWithDeadTabletServer.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "38a585c4199f4a8982834c0d9801452a" format_stamp: "Formatted at 2019-06-09 06:56:51 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:51.508127 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s I0609 06:56:51.509426 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:51.515573 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:51.515744 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestWriteWithDeadTabletServer.1560063404629499-1119/minicluster-data/master-0-root uuid: "38a585c4199f4a8982834c0d9801452a" format_stamp: "Formatted at 2019-06-09 06:56:51 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:51.515957 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestWriteWithDeadTabletServer.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestWriteWithDeadTabletServer.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestWriteWithDeadTabletServer.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:51.576604 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:46433 I0609 06:56:51.578291 1119 webserver.cc:374] Webserver started at http://127.1.23.254:46625/ using document root and password file I0609 06:56:51.579835 2019 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:51.587074 2019 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a: Bootstrap starting. I0609 06:56:51.589020 2019 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a: No blocks or log segments found. Creating new log. I0609 06:56:51.591675 2019 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a: No bootstrap required, opened a new log I0609 06:56:51.592789 2019 raft_consensus.cc:340] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a585c4199f4a8982834c0d9801452a" member_type: VOTER } I0609 06:56:51.593084 2019 raft_consensus.cc:366] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:51.593279 2019 raft_consensus.cc:705] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a585c4199f4a8982834c0d9801452a, State: Initialized, Role: FOLLOWER I0609 06:56:51.593719 2019 consensus_queue.cc:229] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a585c4199f4a8982834c0d9801452a" member_type: VOTER } I0609 06:56:51.594079 2019 raft_consensus.cc:380] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:51.594300 2019 raft_consensus.cc:472] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:51.594516 2019 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:51.597517 2019 raft_consensus.cc:494] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a585c4199f4a8982834c0d9801452a" member_type: VOTER } I0609 06:56:51.597915 2019 leader_election.cc:310] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38a585c4199f4a8982834c0d9801452a; no voters: I0609 06:56:51.598984 2021 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:51.599630 2019 leader_election.cc:296] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:51.601119 2021 raft_consensus.cc:667] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [term 1 LEADER]: Becoming Leader. State: Replica: 38a585c4199f4a8982834c0d9801452a, State: Running, Role: LEADER I0609 06:56:51.601471 2021 consensus_queue.cc:206] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a585c4199f4a8982834c0d9801452a" member_type: VOTER } I0609 06:56:51.604307 2023 sys_catalog.cc:338] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38a585c4199f4a8982834c0d9801452a. Latest consensus state: current_term: 1 leader_uuid: "38a585c4199f4a8982834c0d9801452a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a585c4199f4a8982834c0d9801452a" member_type: VOTER } } I0609 06:56:51.604705 2023 sys_catalog.cc:341] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [sys.catalog]: This master's current role is: LEADER I0609 06:56:51.606058 2022 sys_catalog.cc:338] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38a585c4199f4a8982834c0d9801452a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a585c4199f4a8982834c0d9801452a" member_type: VOTER } } I0609 06:56:51.606065 2019 sys_catalog.cc:338] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "38a585c4199f4a8982834c0d9801452a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a585c4199f4a8982834c0d9801452a" member_type: VOTER } } I0609 06:56:51.606698 2022 sys_catalog.cc:341] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [sys.catalog]: This master's current role is: LEADER I0609 06:56:51.606782 2019 sys_catalog.cc:341] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [sys.catalog]: This master's current role is: LEADER I0609 06:56:51.607836 2019 sys_catalog.cc:426] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:51.608006 2027 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:51.609714 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:51.610901 2027 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:51.642449 2027 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:51.643944 2027 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:51.653592 2027 catalog_manager.cc:4293] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a: Generated new TSK 0 I0609 06:56:51.688221 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 92516us I0609 06:56:51.688496 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:51.688616 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:51.688802 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:51.688920 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:51.689834 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestWriteWithDeadTabletServer.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "470ca18b1ee94cb49eee7cc632c7272a" format_stamp: "Formatted at 2019-06-09 06:56:51 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:51.698207 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.006s sys 0.002s I0609 06:56:51.699379 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:51.706202 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s I0609 06:56:51.706336 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestWriteWithDeadTabletServer.1560063404629499-1119/minicluster-data/ts-0-root uuid: "470ca18b1ee94cb49eee7cc632c7272a" format_stamp: "Formatted at 2019-06-09 06:56:51 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:51.706524 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestWriteWithDeadTabletServer.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestWriteWithDeadTabletServer.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestWriteWithDeadTabletServer.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:51.722803 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:51.722935 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:51.758051 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:41067 I0609 06:56:51.759604 1119 webserver.cc:374] Webserver started at http://127.1.23.193:46227/ using document root and password file I0609 06:56:51.770429 2094 heartbeater.cc:324] Connected to a master server at 127.1.23.254:46433 I0609 06:56:51.770632 2094 heartbeater.cc:405] Registering TS with master... I0609 06:56:51.770954 2094 heartbeater.cc:454] Master 127.1.23.254:46433 requested a full tablet report, sending... I0609 06:56:51.772533 1986 ts_manager.cc:161] Registered new tserver with Master: 470ca18b1ee94cb49eee7cc632c7272a (127.1.23.193:41067) I0609 06:56:51.772851 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.011666344s I0609 06:56:51.774472 1986 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55460 I0609 06:56:51.787668 2094 heartbeater.cc:446] Master 127.1.23.254:46433 was elected leader, sending a full tablet report... I0609 06:56:51.791483 1985 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55462: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:51.815562 2061 tablet_service.cc:1016] Processing CreateTablet for tablet 9895ab7d7e7e4c7397ac2059799b2d8a (table=client-testtb [id=7ce2ba23f20c4d0cbeb16d674c43b1c4]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:51.815991 2060 tablet_service.cc:1016] Processing CreateTablet for tablet 22d97a7080d94e4dbc4af93e099b7a6a (table=client-testtb [id=7ce2ba23f20c4d0cbeb16d674c43b1c4]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:51.816520 2061 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 9895ab7d7e7e4c7397ac2059799b2d8a. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:51.816978 2060 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 22d97a7080d94e4dbc4af93e099b7a6a. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:51.826679 2106 tablet_bootstrap.cc:439] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a: Bootstrap starting. I0609 06:56:51.828972 2106 tablet_bootstrap.cc:590] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a: No blocks or log segments found. Creating new log. I0609 06:56:51.831516 2106 tablet_bootstrap.cc:439] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a: No bootstrap required, opened a new log I0609 06:56:51.831805 2106 ts_tablet_manager.cc:1054] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s I0609 06:56:51.832953 2106 raft_consensus.cc:340] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470ca18b1ee94cb49eee7cc632c7272a" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41067 } } I0609 06:56:51.833288 2106 raft_consensus.cc:366] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:51.833613 2106 raft_consensus.cc:705] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470ca18b1ee94cb49eee7cc632c7272a, State: Initialized, Role: FOLLOWER I0609 06:56:51.834107 2106 consensus_queue.cc:229] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470ca18b1ee94cb49eee7cc632c7272a" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41067 } } I0609 06:56:51.834540 2106 raft_consensus.cc:380] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:51.834722 2106 raft_consensus.cc:472] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:51.835011 2106 raft_consensus.cc:2890] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:51.838655 2106 raft_consensus.cc:494] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470ca18b1ee94cb49eee7cc632c7272a" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41067 } } I0609 06:56:51.839089 2106 leader_election.cc:310] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 470ca18b1ee94cb49eee7cc632c7272a; no voters: I0609 06:56:51.840342 2108 raft_consensus.cc:2644] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:51.840960 2106 leader_election.cc:296] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:51.842576 2106 ts_tablet_manager.cc:1082] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a: Time spent starting tablet: real 0.011s user 0.008s sys 0.001s I0609 06:56:51.843160 2106 tablet_bootstrap.cc:439] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a: Bootstrap starting. I0609 06:56:51.845402 2108 raft_consensus.cc:667] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [term 1 LEADER]: Becoming Leader. State: Replica: 470ca18b1ee94cb49eee7cc632c7272a, State: Running, Role: LEADER I0609 06:56:51.845906 2108 consensus_queue.cc:206] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470ca18b1ee94cb49eee7cc632c7272a" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41067 } } I0609 06:56:51.848624 2106 tablet_bootstrap.cc:590] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a: No blocks or log segments found. Creating new log. I0609 06:56:51.851836 2106 tablet_bootstrap.cc:439] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a: No bootstrap required, opened a new log I0609 06:56:51.852295 2106 ts_tablet_manager.cc:1054] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s I0609 06:56:51.852604 1985 catalog_manager.cc:3989] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a reported cstate change: term changed from 0 to 1, leader changed from to 470ca18b1ee94cb49eee7cc632c7272a (127.1.23.193). New cstate: current_term: 1 leader_uuid: "470ca18b1ee94cb49eee7cc632c7272a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470ca18b1ee94cb49eee7cc632c7272a" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41067 } health_report { overall_health: HEALTHY } } } I0609 06:56:51.854084 2106 raft_consensus.cc:340] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470ca18b1ee94cb49eee7cc632c7272a" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41067 } } I0609 06:56:51.854605 2106 raft_consensus.cc:366] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:51.854878 2106 raft_consensus.cc:705] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470ca18b1ee94cb49eee7cc632c7272a, State: Initialized, Role: FOLLOWER I0609 06:56:51.855334 2106 consensus_queue.cc:229] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470ca18b1ee94cb49eee7cc632c7272a" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41067 } } I0609 06:56:51.855746 2106 raft_consensus.cc:380] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:51.855924 2106 raft_consensus.cc:472] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:51.856209 2106 raft_consensus.cc:2890] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:51.860409 2106 raft_consensus.cc:494] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470ca18b1ee94cb49eee7cc632c7272a" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41067 } } I0609 06:56:51.860826 2106 leader_election.cc:310] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 470ca18b1ee94cb49eee7cc632c7272a; no voters: I0609 06:56:51.861284 2106 leader_election.cc:296] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:51.861441 2110 raft_consensus.cc:2644] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:51.861876 2110 raft_consensus.cc:667] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [term 1 LEADER]: Becoming Leader. State: Replica: 470ca18b1ee94cb49eee7cc632c7272a, State: Running, Role: LEADER I0609 06:56:51.862525 2106 ts_tablet_manager.cc:1082] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a: Time spent starting tablet: real 0.010s user 0.006s sys 0.003s I0609 06:56:51.862354 2110 consensus_queue.cc:206] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470ca18b1ee94cb49eee7cc632c7272a" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41067 } } I0609 06:56:51.866209 1986 catalog_manager.cc:3989] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a reported cstate change: term changed from 0 to 1, leader changed from to 470ca18b1ee94cb49eee7cc632c7272a (127.1.23.193). New cstate: current_term: 1 leader_uuid: "470ca18b1ee94cb49eee7cc632c7272a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470ca18b1ee94cb49eee7cc632c7272a" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41067 } health_report { overall_health: HEALTHY } } } I0609 06:56:51.882536 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:51.901790 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:51.902201 1119 tablet_replica.cc:265] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a: stopping tablet replica I0609 06:56:51.902716 1119 raft_consensus.cc:2147] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:51.903034 1119 raft_consensus.cc:2174] T 9895ab7d7e7e4c7397ac2059799b2d8a P 470ca18b1ee94cb49eee7cc632c7272a [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:51.905486 1119 tablet_replica.cc:265] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a: stopping tablet replica I0609 06:56:51.905812 1119 raft_consensus.cc:2147] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:51.906049 1119 raft_consensus.cc:2174] T 22d97a7080d94e4dbc4af93e099b7a6a P 470ca18b1ee94cb49eee7cc632c7272a [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:51.918030 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:51.918164 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:51.918407 1119 tablet_server.cc:169] TabletServer shutdown complete. W0609 06:56:51.928263 2099 meta_cache.cc:237] Tablet 9895ab7d7e7e4c7397ac2059799b2d8a: Replica 470ca18b1ee94cb49eee7cc632c7272a (127.1.23.193:41067) has failed: Network error: Client connection negotiation failed: client connection to 127.1.23.193:41067: connect: Connection refused (error 111) W0609 06:56:52.940018 2099 batcher.cc:367] Timed out: Failed to write batch of 1 ops to tablet 9895ab7d7e7e4c7397ac2059799b2d8a after 40 attempt(s): Failed to write to server: (no server available): Write(tablet: 9895ab7d7e7e4c7397ac2059799b2d8a, num_ops: 1, num_attempts: 40) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.1.23.193:41067: connect: Connection refused (error 111) I0609 06:56:52.941558 1119 master.cc:247] Master@127.1.23.254:46433 shutting down... I0609 06:56:52.958446 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:52.958724 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:52.958909 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 38a585c4199f4a8982834c0d9801452a: stopping tablet replica I0609 06:56:52.975193 1119 master.cc:258] Master@127.1.23.254:46433 shutdown complete. [ OK ] ClientTest.TestWriteWithDeadTabletServer (1501 ms) [ RUN ] ClientTest.TestSessionMutationBufferMaxNum I0609 06:56:52.988070 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:37263 I0609 06:56:53.000104 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 93016us I0609 06:56:53.000358 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:53.000478 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:53.000661 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:53.000768 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:53.001637 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestSessionMutationBufferMaxNum.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "9122c1a08d374411aa21d7987b6ab42b" format_stamp: "Formatted at 2019-06-09 06:56:52 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:53.010865 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.009s user 0.008s sys 0.001s I0609 06:56:53.012234 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:53.019090 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:53.019224 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestSessionMutationBufferMaxNum.1560063404629499-1119/minicluster-data/master-0-root uuid: "9122c1a08d374411aa21d7987b6ab42b" format_stamp: "Formatted at 2019-06-09 06:56:52 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:53.019419 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestSessionMutationBufferMaxNum.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestSessionMutationBufferMaxNum.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestSessionMutationBufferMaxNum.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:53.103996 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:37263 I0609 06:56:53.105661 1119 webserver.cc:374] Webserver started at http://127.1.23.254:34591/ using document root and password file I0609 06:56:53.107272 2170 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:53.114392 2170 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b: Bootstrap starting. I0609 06:56:53.116472 2170 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b: No blocks or log segments found. Creating new log. I0609 06:56:53.119324 2170 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b: No bootstrap required, opened a new log I0609 06:56:53.120534 2170 raft_consensus.cc:340] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9122c1a08d374411aa21d7987b6ab42b" member_type: VOTER } I0609 06:56:53.120883 2170 raft_consensus.cc:366] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:53.121057 2170 raft_consensus.cc:705] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9122c1a08d374411aa21d7987b6ab42b, State: Initialized, Role: FOLLOWER I0609 06:56:53.121484 2170 consensus_queue.cc:229] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9122c1a08d374411aa21d7987b6ab42b" member_type: VOTER } I0609 06:56:53.121856 2170 raft_consensus.cc:380] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:53.122045 2170 raft_consensus.cc:472] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:53.122272 2170 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:53.125351 2170 raft_consensus.cc:494] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9122c1a08d374411aa21d7987b6ab42b" member_type: VOTER } I0609 06:56:53.125778 2170 leader_election.cc:310] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9122c1a08d374411aa21d7987b6ab42b; no voters: I0609 06:56:53.126890 2172 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:53.127522 2170 leader_election.cc:296] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:53.129016 2172 raft_consensus.cc:667] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [term 1 LEADER]: Becoming Leader. State: Replica: 9122c1a08d374411aa21d7987b6ab42b, State: Running, Role: LEADER I0609 06:56:53.129433 2172 consensus_queue.cc:206] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9122c1a08d374411aa21d7987b6ab42b" member_type: VOTER } I0609 06:56:53.132128 2174 sys_catalog.cc:338] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9122c1a08d374411aa21d7987b6ab42b. Latest consensus state: current_term: 1 leader_uuid: "9122c1a08d374411aa21d7987b6ab42b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9122c1a08d374411aa21d7987b6ab42b" member_type: VOTER } } I0609 06:56:53.132448 2174 sys_catalog.cc:341] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [sys.catalog]: This master's current role is: LEADER I0609 06:56:53.134302 2173 sys_catalog.cc:338] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9122c1a08d374411aa21d7987b6ab42b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9122c1a08d374411aa21d7987b6ab42b" member_type: VOTER } } I0609 06:56:53.134388 2170 sys_catalog.cc:338] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "9122c1a08d374411aa21d7987b6ab42b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9122c1a08d374411aa21d7987b6ab42b" member_type: VOTER } } I0609 06:56:53.134871 2173 sys_catalog.cc:341] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [sys.catalog]: This master's current role is: LEADER I0609 06:56:53.134901 2170 sys_catalog.cc:341] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [sys.catalog]: This master's current role is: LEADER I0609 06:56:53.135974 2170 sys_catalog.cc:426] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:53.136152 2178 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:53.137127 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:53.138765 2178 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:53.175945 2178 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:53.177417 2178 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:53.192168 2178 catalog_manager.cc:4293] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b: Generated new TSK 0 I0609 06:56:53.215777 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 93516us I0609 06:56:53.216050 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:53.216172 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:53.216347 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:53.216452 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:53.217268 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestSessionMutationBufferMaxNum.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "3b226cba4d584a63b5fc00f97c212dbe" format_stamp: "Formatted at 2019-06-09 06:56:53 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:53.226217 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.009s user 0.006s sys 0.002s I0609 06:56:53.227458 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:53.234400 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:53.234532 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestSessionMutationBufferMaxNum.1560063404629499-1119/minicluster-data/ts-0-root uuid: "3b226cba4d584a63b5fc00f97c212dbe" format_stamp: "Formatted at 2019-06-09 06:56:53 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:53.234719 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestSessionMutationBufferMaxNum.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestSessionMutationBufferMaxNum.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestSessionMutationBufferMaxNum.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:53.279948 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:53.280161 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:53.315114 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:40877 I0609 06:56:53.316695 1119 webserver.cc:374] Webserver started at http://127.1.23.193:34861/ using document root and password file I0609 06:56:53.329213 2245 heartbeater.cc:324] Connected to a master server at 127.1.23.254:37263 I0609 06:56:53.329519 2245 heartbeater.cc:405] Registering TS with master... I0609 06:56:53.329892 2245 heartbeater.cc:454] Master 127.1.23.254:37263 requested a full tablet report, sending... I0609 06:56:53.331238 2137 ts_manager.cc:161] Registered new tserver with Master: 3b226cba4d584a63b5fc00f97c212dbe (127.1.23.193:40877) I0609 06:56:53.332054 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.01392008s I0609 06:56:53.333570 2137 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38340 I0609 06:56:53.346671 2245 heartbeater.cc:446] Master 127.1.23.254:37263 was elected leader, sending a full tablet report... I0609 06:56:53.349614 2137 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38342: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:53.378010 2212 tablet_service.cc:1016] Processing CreateTablet for tablet db4b686526ba4bbea1d81be0c1380c51 (table=client-testtb [id=ecbfd7fb2a4e4cfb8cd0622ee911b207]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:53.378332 2211 tablet_service.cc:1016] Processing CreateTablet for tablet 599f6a3f2f704687ae78444e7d7c10e4 (table=client-testtb [id=ecbfd7fb2a4e4cfb8cd0622ee911b207]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:53.378942 2212 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet db4b686526ba4bbea1d81be0c1380c51. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:53.379443 2211 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 599f6a3f2f704687ae78444e7d7c10e4. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:53.392004 2257 tablet_bootstrap.cc:439] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe: Bootstrap starting. I0609 06:56:53.394526 2257 tablet_bootstrap.cc:590] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe: No blocks or log segments found. Creating new log. I0609 06:56:53.397249 2257 tablet_bootstrap.cc:439] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe: No bootstrap required, opened a new log I0609 06:56:53.397665 2257 ts_tablet_manager.cc:1054] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s I0609 06:56:53.398859 2257 raft_consensus.cc:340] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b226cba4d584a63b5fc00f97c212dbe" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 40877 } } I0609 06:56:53.399232 2257 raft_consensus.cc:366] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:53.399415 2257 raft_consensus.cc:705] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b226cba4d584a63b5fc00f97c212dbe, State: Initialized, Role: FOLLOWER I0609 06:56:53.399854 2257 consensus_queue.cc:229] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b226cba4d584a63b5fc00f97c212dbe" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 40877 } } I0609 06:56:53.400290 2257 raft_consensus.cc:380] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:53.400480 2257 raft_consensus.cc:472] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:53.400710 2257 raft_consensus.cc:2890] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:53.403947 2257 raft_consensus.cc:494] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b226cba4d584a63b5fc00f97c212dbe" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 40877 } } I0609 06:56:53.404373 2257 leader_election.cc:310] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b226cba4d584a63b5fc00f97c212dbe; no voters: I0609 06:56:53.405369 2257 leader_election.cc:296] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:53.405532 2259 raft_consensus.cc:2644] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:53.407169 2257 ts_tablet_manager.cc:1082] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s I0609 06:56:53.407060 2259 raft_consensus.cc:667] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [term 1 LEADER]: Becoming Leader. State: Replica: 3b226cba4d584a63b5fc00f97c212dbe, State: Running, Role: LEADER I0609 06:56:53.407737 2257 tablet_bootstrap.cc:439] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe: Bootstrap starting. I0609 06:56:53.407657 2259 consensus_queue.cc:206] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b226cba4d584a63b5fc00f97c212dbe" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 40877 } } I0609 06:56:53.410418 2257 tablet_bootstrap.cc:590] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe: No blocks or log segments found. Creating new log. I0609 06:56:53.412521 2136 catalog_manager.cc:3989] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe reported cstate change: term changed from 0 to 1, leader changed from to 3b226cba4d584a63b5fc00f97c212dbe (127.1.23.193). New cstate: current_term: 1 leader_uuid: "3b226cba4d584a63b5fc00f97c212dbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b226cba4d584a63b5fc00f97c212dbe" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 40877 } health_report { overall_health: HEALTHY } } } I0609 06:56:53.414151 2257 tablet_bootstrap.cc:439] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe: No bootstrap required, opened a new log I0609 06:56:53.414561 2257 ts_tablet_manager.cc:1054] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s I0609 06:56:53.415627 2257 raft_consensus.cc:340] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b226cba4d584a63b5fc00f97c212dbe" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 40877 } } I0609 06:56:53.415978 2257 raft_consensus.cc:366] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:53.416167 2257 raft_consensus.cc:705] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b226cba4d584a63b5fc00f97c212dbe, State: Initialized, Role: FOLLOWER I0609 06:56:53.416739 2257 consensus_queue.cc:229] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b226cba4d584a63b5fc00f97c212dbe" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 40877 } } I0609 06:56:53.417372 2257 raft_consensus.cc:380] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:53.417624 2257 raft_consensus.cc:472] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:53.417949 2257 raft_consensus.cc:2890] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:53.422811 2257 raft_consensus.cc:494] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b226cba4d584a63b5fc00f97c212dbe" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 40877 } } I0609 06:56:53.423235 2257 leader_election.cc:310] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b226cba4d584a63b5fc00f97c212dbe; no voters: I0609 06:56:53.423718 2257 leader_election.cc:296] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:53.423883 2261 raft_consensus.cc:2644] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:53.424391 2261 raft_consensus.cc:667] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [term 1 LEADER]: Becoming Leader. State: Replica: 3b226cba4d584a63b5fc00f97c212dbe, State: Running, Role: LEADER I0609 06:56:53.424791 2257 ts_tablet_manager.cc:1082] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s I0609 06:56:53.424906 2261 consensus_queue.cc:206] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b226cba4d584a63b5fc00f97c212dbe" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 40877 } } I0609 06:56:53.428547 2137 catalog_manager.cc:3989] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe reported cstate change: term changed from 0 to 1, leader changed from to 3b226cba4d584a63b5fc00f97c212dbe (127.1.23.193). New cstate: current_term: 1 leader_uuid: "3b226cba4d584a63b5fc00f97c212dbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b226cba4d584a63b5fc00f97c212dbe" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 40877 } health_report { overall_health: HEALTHY } } } I0609 06:56:54.691735 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:54.712836 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:54.713068 1119 tablet_replica.cc:265] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe: stopping tablet replica I0609 06:56:54.713524 1119 raft_consensus.cc:2147] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:54.713776 1119 raft_consensus.cc:2174] T db4b686526ba4bbea1d81be0c1380c51 P 3b226cba4d584a63b5fc00f97c212dbe [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:54.715572 1119 tablet_replica.cc:265] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe: stopping tablet replica I0609 06:56:54.715939 1119 raft_consensus.cc:2147] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:54.716163 1119 raft_consensus.cc:2174] T 599f6a3f2f704687ae78444e7d7c10e4 P 3b226cba4d584a63b5fc00f97c212dbe [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:54.744629 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:54.744868 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:54.745291 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:54.756929 1119 master.cc:247] Master@127.1.23.254:37263 shutting down... I0609 06:56:54.777746 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:54.778216 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:54.778475 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 9122c1a08d374411aa21d7987b6ab42b: stopping tablet replica I0609 06:56:54.795578 1119 master.cc:258] Master@127.1.23.254:37263 shutdown complete. [ OK ] ClientTest.TestSessionMutationBufferMaxNum (1826 ms) [ RUN ] ClientTest.TestAutoFlushBackgroundPreFlush I0609 06:56:54.814160 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:42483 I0609 06:56:54.825542 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 94016us I0609 06:56:54.825847 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:54.825974 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:54.826151 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:54.826270 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:54.827168 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestAutoFlushBackgroundPreFlush.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "38d853691b6846ccbd2fc27b95f691c9" format_stamp: "Formatted at 2019-06-09 06:56:54 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:54.835366 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s I0609 06:56:54.836542 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:54.842785 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:54.842938 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestAutoFlushBackgroundPreFlush.1560063404629499-1119/minicluster-data/master-0-root uuid: "38d853691b6846ccbd2fc27b95f691c9" format_stamp: "Formatted at 2019-06-09 06:56:54 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:54.843120 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestAutoFlushBackgroundPreFlush.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestAutoFlushBackgroundPreFlush.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestAutoFlushBackgroundPreFlush.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:54.902441 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:42483 I0609 06:56:54.904448 1119 webserver.cc:374] Webserver started at http://127.1.23.254:42303/ using document root and password file I0609 06:56:54.906330 2334 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:54.915817 2334 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9: Bootstrap starting. I0609 06:56:54.918292 2334 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9: No blocks or log segments found. Creating new log. I0609 06:56:54.921080 2334 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9: No bootstrap required, opened a new log I0609 06:56:54.922582 2334 raft_consensus.cc:340] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d853691b6846ccbd2fc27b95f691c9" member_type: VOTER } I0609 06:56:54.922989 2334 raft_consensus.cc:366] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:54.923189 2334 raft_consensus.cc:705] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38d853691b6846ccbd2fc27b95f691c9, State: Initialized, Role: FOLLOWER I0609 06:56:54.923718 2334 consensus_queue.cc:229] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d853691b6846ccbd2fc27b95f691c9" member_type: VOTER } I0609 06:56:54.924113 2334 raft_consensus.cc:380] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:54.924317 2334 raft_consensus.cc:472] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:54.924540 2334 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:54.927861 2334 raft_consensus.cc:494] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d853691b6846ccbd2fc27b95f691c9" member_type: VOTER } I0609 06:56:54.928267 2334 leader_election.cc:310] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38d853691b6846ccbd2fc27b95f691c9; no voters: I0609 06:56:54.929318 2334 leader_election.cc:296] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:54.929615 2336 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:54.930770 2336 raft_consensus.cc:667] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [term 1 LEADER]: Becoming Leader. State: Replica: 38d853691b6846ccbd2fc27b95f691c9, State: Running, Role: LEADER I0609 06:56:54.931124 2336 consensus_queue.cc:206] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d853691b6846ccbd2fc27b95f691c9" member_type: VOTER } I0609 06:56:54.933790 2334 sys_catalog.cc:338] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "38d853691b6846ccbd2fc27b95f691c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d853691b6846ccbd2fc27b95f691c9" member_type: VOTER } } I0609 06:56:54.934350 2334 sys_catalog.cc:341] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [sys.catalog]: This master's current role is: LEADER I0609 06:56:54.934423 2338 sys_catalog.cc:338] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38d853691b6846ccbd2fc27b95f691c9. Latest consensus state: current_term: 1 leader_uuid: "38d853691b6846ccbd2fc27b95f691c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d853691b6846ccbd2fc27b95f691c9" member_type: VOTER } } I0609 06:56:54.934491 2337 sys_catalog.cc:338] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38d853691b6846ccbd2fc27b95f691c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d853691b6846ccbd2fc27b95f691c9" member_type: VOTER } } I0609 06:56:54.935005 2338 sys_catalog.cc:341] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [sys.catalog]: This master's current role is: LEADER I0609 06:56:54.935230 2337 sys_catalog.cc:341] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [sys.catalog]: This master's current role is: LEADER I0609 06:56:54.940189 2334 sys_catalog.cc:426] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:54.940953 2341 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:54.941241 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:54.943763 2341 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:54.983448 2341 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:54.984863 2341 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:54.994626 2341 catalog_manager.cc:4293] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9: Generated new TSK 0 I0609 06:56:55.020323 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 94516us I0609 06:56:55.020609 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:55.020737 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:55.020908 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:55.021013 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:55.021920 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestAutoFlushBackgroundPreFlush.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" format_stamp: "Formatted at 2019-06-09 06:56:55 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:55.030946 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.009s user 0.009s sys 0.000s I0609 06:56:55.032205 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:55.039130 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:55.039273 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestAutoFlushBackgroundPreFlush.1560063404629499-1119/minicluster-data/ts-0-root uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" format_stamp: "Formatted at 2019-06-09 06:56:55 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:55.039510 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestAutoFlushBackgroundPreFlush.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestAutoFlushBackgroundPreFlush.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestAutoFlushBackgroundPreFlush.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:55.135537 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:55.135679 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:55.179004 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:46795 I0609 06:56:55.180620 1119 webserver.cc:374] Webserver started at http://127.1.23.193:37303/ using document root and password file I0609 06:56:55.197283 2408 heartbeater.cc:324] Connected to a master server at 127.1.23.254:42483 I0609 06:56:55.197546 2408 heartbeater.cc:405] Registering TS with master... I0609 06:56:55.197916 2408 heartbeater.cc:454] Master 127.1.23.254:42483 requested a full tablet report, sending... I0609 06:56:55.199301 2301 ts_manager.cc:161] Registered new tserver with Master: bd2c52ff9e9442ff954c8cfa1321e7a6 (127.1.23.193:46795) I0609 06:56:55.199384 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.017139685s I0609 06:56:55.201469 2301 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50756 I0609 06:56:55.213845 2408 heartbeater.cc:446] Master 127.1.23.254:42483 was elected leader, sending a full tablet report... I0609 06:56:55.219559 2300 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50758: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:55.249814 2375 tablet_service.cc:1016] Processing CreateTablet for tablet 0ebea029b13a4906899db6e324571ea1 (table=client-testtb [id=9cfba1e02f454d7fb9d483d338ec4398]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:55.250025 2374 tablet_service.cc:1016] Processing CreateTablet for tablet 4d86f614f1264e65b774caf3946587ae (table=client-testtb [id=9cfba1e02f454d7fb9d483d338ec4398]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:55.250558 2375 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 0ebea029b13a4906899db6e324571ea1. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:55.250977 2374 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 4d86f614f1264e65b774caf3946587ae. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:55.264420 2420 tablet_bootstrap.cc:439] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6: Bootstrap starting. I0609 06:56:55.267530 2420 tablet_bootstrap.cc:590] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6: No blocks or log segments found. Creating new log. I0609 06:56:55.270350 2420 tablet_bootstrap.cc:439] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6: No bootstrap required, opened a new log I0609 06:56:55.270645 2420 ts_tablet_manager.cc:1054] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s I0609 06:56:55.272452 2420 raft_consensus.cc:340] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46795 } } I0609 06:56:55.272856 2420 raft_consensus.cc:366] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:55.273025 2420 raft_consensus.cc:705] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd2c52ff9e9442ff954c8cfa1321e7a6, State: Initialized, Role: FOLLOWER I0609 06:56:55.273488 2420 consensus_queue.cc:229] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46795 } } I0609 06:56:55.273919 2420 raft_consensus.cc:380] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:55.274101 2420 raft_consensus.cc:472] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:55.274327 2420 raft_consensus.cc:2890] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:55.278106 2420 raft_consensus.cc:494] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46795 } } I0609 06:56:55.278682 2420 leader_election.cc:310] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd2c52ff9e9442ff954c8cfa1321e7a6; no voters: I0609 06:56:55.279688 2420 leader_election.cc:296] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:55.279959 2422 raft_consensus.cc:2644] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:55.281232 2422 raft_consensus.cc:667] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 1 LEADER]: Becoming Leader. State: Replica: bd2c52ff9e9442ff954c8cfa1321e7a6, State: Running, Role: LEADER I0609 06:56:55.282130 2422 consensus_queue.cc:206] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46795 } } I0609 06:56:55.282706 2420 ts_tablet_manager.cc:1082] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s I0609 06:56:55.283138 2420 tablet_bootstrap.cc:439] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6: Bootstrap starting. I0609 06:56:55.286754 2420 tablet_bootstrap.cc:590] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6: No blocks or log segments found. Creating new log. I0609 06:56:55.288575 2300 catalog_manager.cc:3989] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 reported cstate change: term changed from 0 to 1, leader changed from to bd2c52ff9e9442ff954c8cfa1321e7a6 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46795 } health_report { overall_health: HEALTHY } } } I0609 06:56:55.290431 2420 tablet_bootstrap.cc:439] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6: No bootstrap required, opened a new log I0609 06:56:55.290711 2420 ts_tablet_manager.cc:1054] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s I0609 06:56:55.291873 2420 raft_consensus.cc:340] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46795 } } I0609 06:56:55.292291 2420 raft_consensus.cc:366] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:55.292552 2420 raft_consensus.cc:705] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd2c52ff9e9442ff954c8cfa1321e7a6, State: Initialized, Role: FOLLOWER I0609 06:56:55.293223 2420 consensus_queue.cc:229] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46795 } } I0609 06:56:55.293848 2420 raft_consensus.cc:380] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:55.294131 2420 raft_consensus.cc:472] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:55.294428 2420 raft_consensus.cc:2890] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:55.298221 2420 raft_consensus.cc:494] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46795 } } I0609 06:56:55.298660 2420 leader_election.cc:310] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd2c52ff9e9442ff954c8cfa1321e7a6; no voters: I0609 06:56:55.299080 2420 leader_election.cc:296] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:55.299201 2422 raft_consensus.cc:2644] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:55.299662 2422 raft_consensus.cc:667] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 1 LEADER]: Becoming Leader. State: Replica: bd2c52ff9e9442ff954c8cfa1321e7a6, State: Running, Role: LEADER I0609 06:56:55.299993 2420 ts_tablet_manager.cc:1082] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6: Time spent starting tablet: real 0.009s user 0.005s sys 0.005s I0609 06:56:55.300102 2422 consensus_queue.cc:206] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46795 } } I0609 06:56:55.303702 2300 catalog_manager.cc:3989] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 reported cstate change: term changed from 0 to 1, leader changed from to bd2c52ff9e9442ff954c8cfa1321e7a6 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c52ff9e9442ff954c8cfa1321e7a6" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46795 } health_report { overall_health: HEALTHY } } } I0609 06:56:55.369082 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:55.401268 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:55.401836 1119 tablet_replica.cc:265] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6: stopping tablet replica I0609 06:56:55.402617 1119 raft_consensus.cc:2147] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:55.403036 1119 raft_consensus.cc:2174] T 0ebea029b13a4906899db6e324571ea1 P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:55.406116 1119 tablet_replica.cc:265] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6: stopping tablet replica I0609 06:56:55.406512 1119 raft_consensus.cc:2147] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:55.406846 1119 raft_consensus.cc:2174] T 4d86f614f1264e65b774caf3946587ae P bd2c52ff9e9442ff954c8cfa1321e7a6 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:55.428501 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:55.428624 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:55.428915 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:55.435883 1119 master.cc:247] Master@127.1.23.254:42483 shutting down... I0609 06:56:55.456471 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:55.456759 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:55.457084 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 38d853691b6846ccbd2fc27b95f691c9: stopping tablet replica I0609 06:56:55.468509 1119 master.cc:258] Master@127.1.23.254:42483 shutdown complete. [ OK ] ClientTest.TestAutoFlushBackgroundPreFlush (670 ms) [ RUN ] ClientTest.TestBasicAlterOperations I0609 06:56:55.483875 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:35257 I0609 06:56:55.496551 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 94516us I0609 06:56:55.496848 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:55.497001 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:55.497233 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:55.497470 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:55.498409 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBasicAlterOperations.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "da9e629263c34a098be6bdb73a3157ff" format_stamp: "Formatted at 2019-06-09 06:56:55 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:55.507247 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.008s sys 0.001s I0609 06:56:55.508569 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:55.515172 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:55.515327 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBasicAlterOperations.1560063404629499-1119/minicluster-data/master-0-root uuid: "da9e629263c34a098be6bdb73a3157ff" format_stamp: "Formatted at 2019-06-09 06:56:55 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:55.515560 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBasicAlterOperations.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBasicAlterOperations.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBasicAlterOperations.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:55.610483 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:35257 I0609 06:56:55.612114 1119 webserver.cc:374] Webserver started at http://127.1.23.254:37049/ using document root and password file I0609 06:56:55.614166 2486 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:55.625933 2486 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: Bootstrap starting. I0609 06:56:55.628911 2486 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: No blocks or log segments found. Creating new log. I0609 06:56:55.632352 2486 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: No bootstrap required, opened a new log I0609 06:56:55.634089 2486 raft_consensus.cc:340] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9e629263c34a098be6bdb73a3157ff" member_type: VOTER } I0609 06:56:55.634553 2486 raft_consensus.cc:366] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:55.634784 2486 raft_consensus.cc:705] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da9e629263c34a098be6bdb73a3157ff, State: Initialized, Role: FOLLOWER I0609 06:56:55.635339 2486 consensus_queue.cc:229] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9e629263c34a098be6bdb73a3157ff" member_type: VOTER } I0609 06:56:55.635864 2486 raft_consensus.cc:380] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:55.636104 2486 raft_consensus.cc:472] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:55.636399 2486 raft_consensus.cc:2890] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:55.641743 2486 raft_consensus.cc:494] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9e629263c34a098be6bdb73a3157ff" member_type: VOTER } I0609 06:56:55.642268 2486 leader_election.cc:310] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da9e629263c34a098be6bdb73a3157ff; no voters: I0609 06:56:55.643508 2486 leader_election.cc:296] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:55.643697 2488 raft_consensus.cc:2644] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:55.645584 2488 raft_consensus.cc:667] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [term 1 LEADER]: Becoming Leader. State: Replica: da9e629263c34a098be6bdb73a3157ff, State: Running, Role: LEADER I0609 06:56:55.646133 2488 consensus_queue.cc:206] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9e629263c34a098be6bdb73a3157ff" member_type: VOTER } I0609 06:56:55.648115 2486 sys_catalog.cc:338] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "da9e629263c34a098be6bdb73a3157ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9e629263c34a098be6bdb73a3157ff" member_type: VOTER } } I0609 06:56:55.649412 2486 sys_catalog.cc:341] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [sys.catalog]: This master's current role is: LEADER I0609 06:56:55.648353 2489 sys_catalog.cc:338] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da9e629263c34a098be6bdb73a3157ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9e629263c34a098be6bdb73a3157ff" member_type: VOTER } } I0609 06:56:55.648830 2490 sys_catalog.cc:338] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader da9e629263c34a098be6bdb73a3157ff. Latest consensus state: current_term: 1 leader_uuid: "da9e629263c34a098be6bdb73a3157ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9e629263c34a098be6bdb73a3157ff" member_type: VOTER } } I0609 06:56:55.649930 2489 sys_catalog.cc:341] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [sys.catalog]: This master's current role is: LEADER I0609 06:56:55.650899 2490 sys_catalog.cc:341] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [sys.catalog]: This master's current role is: LEADER I0609 06:56:55.652220 2486 sys_catalog.cc:426] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:55.652431 2493 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:55.653720 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:55.655441 2493 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:55.729080 2493 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:55.730547 2493 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:55.739161 2493 catalog_manager.cc:4293] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: Generated new TSK 0 I0609 06:56:55.806963 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 94516us I0609 06:56:55.807466 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:55.807684 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:55.807957 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:55.808123 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:55.809389 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBasicAlterOperations.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "5c32c752daa24dd482c8455f2e4a158c" format_stamp: "Formatted at 2019-06-09 06:56:55 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:55.826573 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.017s user 0.012s sys 0.005s I0609 06:56:55.828042 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:55.838193 1119 fs_manager.cc:419] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s I0609 06:56:55.838378 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBasicAlterOperations.1560063404629499-1119/minicluster-data/ts-0-root uuid: "5c32c752daa24dd482c8455f2e4a158c" format_stamp: "Formatted at 2019-06-09 06:56:55 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:55.838613 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBasicAlterOperations.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBasicAlterOperations.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBasicAlterOperations.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:55.894070 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:55.894265 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:56.008052 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:45725 I0609 06:56:56.009774 1119 webserver.cc:374] Webserver started at http://127.1.23.193:35633/ using document root and password file I0609 06:56:56.023453 2560 heartbeater.cc:324] Connected to a master server at 127.1.23.254:35257 I0609 06:56:56.023806 2560 heartbeater.cc:405] Registering TS with master... I0609 06:56:56.024282 2560 heartbeater.cc:454] Master 127.1.23.254:35257 requested a full tablet report, sending... I0609 06:56:56.025841 2453 ts_manager.cc:161] Registered new tserver with Master: 5c32c752daa24dd482c8455f2e4a158c (127.1.23.193:45725) I0609 06:56:56.026337 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.015201828s I0609 06:56:56.028261 2453 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42998 I0609 06:56:56.042152 2560 heartbeater.cc:446] Master 127.1.23.254:35257 was elected leader, sending a full tablet report... I0609 06:56:56.047336 2453 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43000: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:56.075145 2527 tablet_service.cc:1016] Processing CreateTablet for tablet f4cb1f8a0e404a918d16e7179fd08a04 (table=client-testtb [id=0681d8e9404c400599d8020786aad7fb]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:56.075469 2526 tablet_service.cc:1016] Processing CreateTablet for tablet c79f64bd0cc346f586d0baee5ff19d6c (table=client-testtb [id=0681d8e9404c400599d8020786aad7fb]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:56.076032 2527 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet f4cb1f8a0e404a918d16e7179fd08a04. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:56.076524 2526 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet c79f64bd0cc346f586d0baee5ff19d6c. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:56.086443 2572 tablet_bootstrap.cc:439] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: Bootstrap starting. I0609 06:56:56.089865 2572 tablet_bootstrap.cc:590] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: No blocks or log segments found. Creating new log. I0609 06:56:56.092811 2572 tablet_bootstrap.cc:439] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: No bootstrap required, opened a new log I0609 06:56:56.093255 2572 ts_tablet_manager.cc:1054] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s I0609 06:56:56.095180 2572 raft_consensus.cc:340] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c32c752daa24dd482c8455f2e4a158c" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45725 } } I0609 06:56:56.095721 2572 raft_consensus.cc:366] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:56.095980 2572 raft_consensus.cc:705] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c32c752daa24dd482c8455f2e4a158c, State: Initialized, Role: FOLLOWER I0609 06:56:56.096491 2572 consensus_queue.cc:229] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c32c752daa24dd482c8455f2e4a158c" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45725 } } I0609 06:56:56.097090 2572 raft_consensus.cc:380] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:56.097378 2572 raft_consensus.cc:472] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:56.097687 2572 raft_consensus.cc:2890] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:56.102627 2572 raft_consensus.cc:494] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c32c752daa24dd482c8455f2e4a158c" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45725 } } I0609 06:56:56.103235 2572 leader_election.cc:310] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c32c752daa24dd482c8455f2e4a158c; no voters: I0609 06:56:56.104579 2572 leader_election.cc:296] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:56.104876 2574 raft_consensus.cc:2644] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:56.107226 2572 ts_tablet_manager.cc:1082] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s I0609 06:56:56.107807 2572 tablet_bootstrap.cc:439] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: Bootstrap starting. I0609 06:56:56.110421 2572 tablet_bootstrap.cc:590] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: No blocks or log segments found. Creating new log. I0609 06:56:56.112905 2574 raft_consensus.cc:667] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [term 1 LEADER]: Becoming Leader. State: Replica: 5c32c752daa24dd482c8455f2e4a158c, State: Running, Role: LEADER I0609 06:56:56.113648 2574 consensus_queue.cc:206] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c32c752daa24dd482c8455f2e4a158c" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45725 } } I0609 06:56:56.114984 2572 tablet_bootstrap.cc:439] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: No bootstrap required, opened a new log I0609 06:56:56.115377 2572 ts_tablet_manager.cc:1054] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s I0609 06:56:56.118688 2572 raft_consensus.cc:340] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c32c752daa24dd482c8455f2e4a158c" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45725 } } I0609 06:56:56.119266 2572 raft_consensus.cc:366] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:56.119554 2572 raft_consensus.cc:705] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c32c752daa24dd482c8455f2e4a158c, State: Initialized, Role: FOLLOWER I0609 06:56:56.120077 2453 catalog_manager.cc:3989] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c reported cstate change: term changed from 0 to 1, leader changed from to 5c32c752daa24dd482c8455f2e4a158c (127.1.23.193). New cstate: current_term: 1 leader_uuid: "5c32c752daa24dd482c8455f2e4a158c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c32c752daa24dd482c8455f2e4a158c" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45725 } health_report { overall_health: HEALTHY } } } I0609 06:56:56.120291 2572 consensus_queue.cc:229] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c32c752daa24dd482c8455f2e4a158c" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45725 } } I0609 06:56:56.121459 2572 raft_consensus.cc:380] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:56.122272 2572 raft_consensus.cc:472] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:56.122628 2572 raft_consensus.cc:2890] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:56.127966 2572 raft_consensus.cc:494] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c32c752daa24dd482c8455f2e4a158c" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45725 } } I0609 06:56:56.128607 2572 leader_election.cc:310] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c32c752daa24dd482c8455f2e4a158c; no voters: I0609 06:56:56.129369 2572 leader_election.cc:296] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:56.129531 2574 raft_consensus.cc:2644] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:56.130219 2574 raft_consensus.cc:667] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [term 1 LEADER]: Becoming Leader. State: Replica: 5c32c752daa24dd482c8455f2e4a158c, State: Running, Role: LEADER I0609 06:56:56.130609 2572 ts_tablet_manager.cc:1082] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: Time spent starting tablet: real 0.015s user 0.010s sys 0.002s I0609 06:56:56.130774 2574 consensus_queue.cc:206] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c32c752daa24dd482c8455f2e4a158c" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45725 } } I0609 06:56:56.135010 2453 catalog_manager.cc:3989] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c reported cstate change: term changed from 0 to 1, leader changed from to 5c32c752daa24dd482c8455f2e4a158c (127.1.23.193). New cstate: current_term: 1 leader_uuid: "5c32c752daa24dd482c8455f2e4a158c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c32c752daa24dd482c8455f2e4a158c" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45725 } health_report { overall_health: HEALTHY } } } I0609 06:56:56.145561 2453 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: RENAME_COLUMN rename_column { old_name: "int_val" new_name: "" } } modify_external_catalogs: true I0609 06:56:56.149161 2453 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: RENAME_COLUMN rename_column { old_name: "int_val" new_name: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" } } modify_external_catalogs: true I0609 06:56:56.153658 2453 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } new_table_name: "" modify_external_catalogs: true I0609 06:56:56.156695 2453 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } new_table_name: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" modify_external_catalogs: true I0609 06:56:56.208568 2453 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c0" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c1" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c2" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c3" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c4" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c5" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c6" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c7" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c8" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c9" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c10" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c11" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c12" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c13" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c14" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c15" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c16" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c17" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c18" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c19" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c20" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c21" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c22" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c23" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c24" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c25" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c26" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c27" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c28" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c29" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c30" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c31" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c32" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c33" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c34" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c35" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c36" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c37" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c38" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c39" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c40" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c41" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c42" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c43" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c44" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c45" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c46" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c47" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c48" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c49" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c50" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c51" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c52" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c53" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c54" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c55" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c56" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c57" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c58" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c59" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c60" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c61" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c62" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c63" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c64" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c65" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c66" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c67" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c68" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c69" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c70" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c71" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c72" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c73" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c74" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c75" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c76" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c77" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c78" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c79" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c80" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c81" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c82" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c83" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c84" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c85" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c86" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c87" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c88" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c89" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c90" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c91" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c92" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c93" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c94" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c95" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c96" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c97" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c98" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c99" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c100" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c101" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c102" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c103" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c104" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c105" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c106" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c107" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c108" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c109" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c110" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c111" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c112" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c113" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c114" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c115" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c116" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c117" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c118" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c119" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c120" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c121" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c122" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c123" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c124" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c125" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c126" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c127" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c128" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c129" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c130" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c131" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c132" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c133" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c134" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c135" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c136" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c137" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c138" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c139" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c140" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c141" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c142" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c143" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c144" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c145" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c146" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c147" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c148" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c149" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c150" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c151" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c152" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c153" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c154" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c155" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c156" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c157" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c158" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c159" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "c160" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } I0609 06:56:56.343549 2452 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "key" type: INT32 is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } modify_external_catalogs: true I0609 06:56:56.347148 2452 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "key" } } modify_external_catalogs: true I0609 06:56:56.350184 2452 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: RENAME_COLUMN rename_column { old_name: "int_val" new_name: "string_val" } } modify_external_catalogs: true I0609 06:56:56.354737 2452 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "int_val" } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "new_col" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } modify_external_catalogs: true I0609 06:56:56.373184 2582 tablet.cc:1254] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key INT32 NOT NULL, 11:int_val INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, PRIMARY KEY (key) ) version 0 to ( 10:key INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, PRIMARY KEY (key) ) version 1 I0609 06:56:56.374811 2583 tablet.cc:1254] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key INT32 NOT NULL, 11:int_val INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, PRIMARY KEY (key) ) version 0 to ( 10:key INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, PRIMARY KEY (key) ) version 1 I0609 06:56:56.375890 2582 tablet.cc:1193] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.376283 2583 tablet.cc:1193] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.392709 2452 catalog_manager.cc:4435] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: client-testtb [id=0681d8e9404c400599d8020786aad7fb] alter complete (version 1) I0609 06:56:56.399370 2452 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "new_string_val" type: STRING is_nullable: true encoding: GROUP_VARINT compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } modify_external_catalogs: true I0609 06:56:56.402513 2452 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "new_string_val" type: STRING is_nullable: true encoding: PREFIX_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } } modify_external_catalogs: true I0609 06:56:56.412845 2582 tablet.cc:1254] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, PRIMARY KEY (key) ) version 1 to ( 10:key INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key) ) version 2 I0609 06:56:56.414341 2582 tablet.cc:1193] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.413914 2583 tablet.cc:1254] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, PRIMARY KEY (key) ) version 1 to ( 10:key INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key) ) version 2 I0609 06:56:56.415241 2583 tablet.cc:1193] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.430668 2453 catalog_manager.cc:4435] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: client-testtb [id=0681d8e9404c400599d8020786aad7fb] alter complete (version 2) I0609 06:56:56.435607 2453 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: RENAME_COLUMN rename_column { old_name: "key" new_name: "key2" } } modify_external_catalogs: true I0609 06:56:56.446105 2582 tablet.cc:1254] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key) ) version 2 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 3 I0609 06:56:56.446370 2583 tablet.cc:1254] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key) ) version 2 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 3 I0609 06:56:56.447492 2582 tablet.cc:1193] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.447799 2583 tablet.cc:1193] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.464740 2453 catalog_manager.cc:4435] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: client-testtb [id=0681d8e9404c400599d8020786aad7fb] alter complete (version 3) I0609 06:56:56.467808 2452 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: ALTER_COLUMN alter_column { delta { name: "string_val" default_value: "hello!" } } } modify_external_catalogs: true I0609 06:56:56.478571 2582 tablet.cc:1254] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 3 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 4 I0609 06:56:56.479820 2582 tablet.cc:1193] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.479605 2583 tablet.cc:1254] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 3 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 4 I0609 06:56:56.480592 2583 tablet.cc:1193] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.495507 2453 catalog_manager.cc:4435] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: client-testtb [id=0681d8e9404c400599d8020786aad7fb] alter complete (version 4) I0609 06:56:56.500807 2453 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: ALTER_COLUMN alter_column { delta { name: "non_null_with_default" default_value: "1\324\000\000\000\000\000\000" } } } modify_external_catalogs: true I0609 06:56:56.510565 2583 tablet.cc:1254] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 4 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 5 I0609 06:56:56.510474 2582 tablet.cc:1254] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 4 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 5 I0609 06:56:56.511835 2583 tablet.cc:1193] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.512060 2582 tablet.cc:1193] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.527508 2453 catalog_manager.cc:4435] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: client-testtb [id=0681d8e9404c400599d8020786aad7fb] alter complete (version 5) I0609 06:56:56.533217 2453 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: ALTER_COLUMN alter_column { delta { name: "string_val" remove_default: true } } } modify_external_catalogs: true I0609 06:56:56.545294 2582 tablet.cc:1254] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 5 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 6 I0609 06:56:56.545437 2583 tablet.cc:1254] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 5 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 6 I0609 06:56:56.546180 2582 tablet.cc:1193] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.546360 2583 tablet.cc:1193] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.562058 2453 catalog_manager.cc:4435] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: client-testtb [id=0681d8e9404c400599d8020786aad7fb] alter complete (version 6) I0609 06:56:56.569167 2453 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: ALTER_COLUMN alter_column { delta { name: "non_null_with_default" remove_default: true } } } modify_external_catalogs: true I0609 06:56:56.580667 2583 tablet.cc:1254] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 6 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 7 I0609 06:56:56.581346 2582 tablet.cc:1254] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 6 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 7 I0609 06:56:56.581985 2583 tablet.cc:1193] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.582499 2582 tablet.cc:1193] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: MemRowSet was empty: no flush needed. I0609 06:56:56.596516 2453 catalog_manager.cc:4435] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: client-testtb [id=0681d8e9404c400599d8020786aad7fb] alter complete (version 7) I0609 06:56:56.600050 2453 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: ALTER_COLUMN alter_column { delta { name: "non_null_with_default" default_value: "aaa" } } } modify_external_catalogs: true I0609 06:56:56.603092 2453 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } alter_schema_steps { type: ALTER_COLUMN alter_column { delta { name: "string_val" encoding: PLAIN_ENCODING compression: LZ4 block_size: 16777216 } } } modify_external_catalogs: true I0609 06:56:56.613448 2583 tablet.cc:1254] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 7 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 8 I0609 06:56:56.615504 2582 tablet.cc:1254] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 7 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 8 I0609 06:56:56.632645 2453 catalog_manager.cc:4435] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: client-testtb [id=0681d8e9404c400599d8020786aad7fb] alter complete (version 8) I0609 06:56:56.635140 2453 catalog_manager.cc:2315] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43000: table { table_name: "client-testtb" } new_table_name: "RenamedTable" modify_external_catalogs: true I0609 06:56:56.645382 2582 tablet.cc:1254] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 8 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 9 I0609 06:56:56.646497 2583 tablet.cc:1254] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: Alter schema from ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 8 to ( 10:key2 INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, 14:new_col INT32 NULLABLE, 15:new_string_val STRING NULLABLE, PRIMARY KEY (key2) ) version 9 I0609 06:56:56.664853 2452 catalog_manager.cc:4435] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: RenamedTable [id=0681d8e9404c400599d8020786aad7fb] alter complete (version 9) I0609 06:56:56.668789 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:56.695801 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:56.696039 1119 tablet_replica.cc:265] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c: stopping tablet replica I0609 06:56:56.696374 1119 raft_consensus.cc:2147] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:56.696647 1119 raft_consensus.cc:2174] T c79f64bd0cc346f586d0baee5ff19d6c P 5c32c752daa24dd482c8455f2e4a158c [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:56.700561 1119 tablet_replica.cc:265] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c: stopping tablet replica I0609 06:56:56.700868 1119 raft_consensus.cc:2147] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:56.701136 1119 raft_consensus.cc:2174] T f4cb1f8a0e404a918d16e7179fd08a04 P 5c32c752daa24dd482c8455f2e4a158c [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:56.723033 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:56.723253 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:56.723606 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:56.731842 1119 master.cc:247] Master@127.1.23.254:35257 shutting down... I0609 06:56:56.752161 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:56.752506 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:56.752821 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P da9e629263c34a098be6bdb73a3157ff: stopping tablet replica I0609 06:56:56.763156 1119 master.cc:258] Master@127.1.23.254:35257 shutdown complete. [ OK ] ClientTest.TestBasicAlterOperations (1300 ms) [ RUN ] ClientTest.TestMasterLookupPermits I0609 06:56:56.784862 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:41285 I0609 06:56:56.809232 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 95016us I0609 06:56:56.809669 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:56.809866 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:56.810115 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:56.810281 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:56.811549 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterLookupPermits.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "9bd45aac38f34cce8c7c85a157a1a38c" format_stamp: "Formatted at 2019-06-09 06:56:56 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:56.825700 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.014s user 0.009s sys 0.006s I0609 06:56:56.828004 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:56.834573 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:56.834717 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterLookupPermits.1560063404629499-1119/minicluster-data/master-0-root uuid: "9bd45aac38f34cce8c7c85a157a1a38c" format_stamp: "Formatted at 2019-06-09 06:56:56 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:56.834913 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterLookupPermits.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterLookupPermits.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterLookupPermits.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:56.898412 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:41285 I0609 06:56:56.900079 1119 webserver.cc:374] Webserver started at http://127.1.23.254:43849/ using document root and password file I0609 06:56:56.901773 2638 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:56.908876 2638 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c: Bootstrap starting. I0609 06:56:56.910918 2638 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c: No blocks or log segments found. Creating new log. I0609 06:56:56.913468 2638 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c: No bootstrap required, opened a new log I0609 06:56:56.914599 2638 raft_consensus.cc:340] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd45aac38f34cce8c7c85a157a1a38c" member_type: VOTER } I0609 06:56:56.914924 2638 raft_consensus.cc:366] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:56.915102 2638 raft_consensus.cc:705] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bd45aac38f34cce8c7c85a157a1a38c, State: Initialized, Role: FOLLOWER I0609 06:56:56.915562 2638 consensus_queue.cc:229] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd45aac38f34cce8c7c85a157a1a38c" member_type: VOTER } I0609 06:56:56.916049 2638 raft_consensus.cc:380] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:56.916224 2638 raft_consensus.cc:472] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:56.916450 2638 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:56.919663 2638 raft_consensus.cc:494] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd45aac38f34cce8c7c85a157a1a38c" member_type: VOTER } I0609 06:56:56.920130 2638 leader_election.cc:310] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bd45aac38f34cce8c7c85a157a1a38c; no voters: I0609 06:56:56.921128 2638 leader_election.cc:296] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:56.921370 2640 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:56.922724 2640 raft_consensus.cc:667] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [term 1 LEADER]: Becoming Leader. State: Replica: 9bd45aac38f34cce8c7c85a157a1a38c, State: Running, Role: LEADER I0609 06:56:56.923164 2640 consensus_queue.cc:206] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd45aac38f34cce8c7c85a157a1a38c" member_type: VOTER } I0609 06:56:56.925799 2638 sys_catalog.cc:338] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "9bd45aac38f34cce8c7c85a157a1a38c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd45aac38f34cce8c7c85a157a1a38c" member_type: VOTER } } I0609 06:56:56.926189 2638 sys_catalog.cc:341] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [sys.catalog]: This master's current role is: LEADER I0609 06:56:56.928038 2642 sys_catalog.cc:338] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bd45aac38f34cce8c7c85a157a1a38c. Latest consensus state: current_term: 1 leader_uuid: "9bd45aac38f34cce8c7c85a157a1a38c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd45aac38f34cce8c7c85a157a1a38c" member_type: VOTER } } I0609 06:56:56.928707 2638 sys_catalog.cc:426] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:56.928494 2641 sys_catalog.cc:338] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bd45aac38f34cce8c7c85a157a1a38c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd45aac38f34cce8c7c85a157a1a38c" member_type: VOTER } } I0609 06:56:56.928972 2646 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:56.928951 2642 sys_catalog.cc:341] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [sys.catalog]: This master's current role is: LEADER I0609 06:56:56.929208 2641 sys_catalog.cc:341] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [sys.catalog]: This master's current role is: LEADER I0609 06:56:56.929960 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:56.931603 2646 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:56.961467 2646 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:56.962934 2646 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:56.976598 2646 catalog_manager.cc:4293] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c: Generated new TSK 0 I0609 06:56:57.007346 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 95516us I0609 06:56:57.007661 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:57.007809 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:57.008026 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:57.008141 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:57.009109 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterLookupPermits.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "9564879d3b7f4a318f5943ce6120319e" format_stamp: "Formatted at 2019-06-09 06:56:57 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:57.017856 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s I0609 06:56:57.019075 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:57.025753 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:57.025882 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterLookupPermits.1560063404629499-1119/minicluster-data/ts-0-root uuid: "9564879d3b7f4a318f5943ce6120319e" format_stamp: "Formatted at 2019-06-09 06:56:57 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:57.026109 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterLookupPermits.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterLookupPermits.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestMasterLookupPermits.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:57.051437 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:57.051570 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:57.093842 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:45367 I0609 06:56:57.095532 1119 webserver.cc:374] Webserver started at http://127.1.23.193:41073/ using document root and password file I0609 06:56:57.106449 2713 heartbeater.cc:324] Connected to a master server at 127.1.23.254:41285 I0609 06:56:57.106654 2713 heartbeater.cc:405] Registering TS with master... I0609 06:56:57.107038 2713 heartbeater.cc:454] Master 127.1.23.254:41285 requested a full tablet report, sending... I0609 06:56:57.108338 2605 ts_manager.cc:161] Registered new tserver with Master: 9564879d3b7f4a318f5943ce6120319e (127.1.23.193:45367) I0609 06:56:57.109068 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.012119082s I0609 06:56:57.109975 2605 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44136 I0609 06:56:57.123500 2713 heartbeater.cc:446] Master 127.1.23.254:41285 was elected leader, sending a full tablet report... I0609 06:56:57.128617 2604 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44138: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:57.154647 2680 tablet_service.cc:1016] Processing CreateTablet for tablet 2450fc3e86c740899f80f39754045222 (table=client-testtb [id=eb96972789004ddf871e8f3d6e7ad421]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:57.154855 2679 tablet_service.cc:1016] Processing CreateTablet for tablet 676c74517fd649188a865c12ae1d49c4 (table=client-testtb [id=eb96972789004ddf871e8f3d6e7ad421]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:57.155570 2680 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 2450fc3e86c740899f80f39754045222. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:57.155970 2679 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 676c74517fd649188a865c12ae1d49c4. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:57.169178 2725 tablet_bootstrap.cc:439] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e: Bootstrap starting. I0609 06:56:57.171564 2725 tablet_bootstrap.cc:590] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e: No blocks or log segments found. Creating new log. I0609 06:56:57.174911 2725 tablet_bootstrap.cc:439] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e: No bootstrap required, opened a new log I0609 06:56:57.175240 2725 ts_tablet_manager.cc:1054] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s I0609 06:56:57.176401 2725 raft_consensus.cc:340] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9564879d3b7f4a318f5943ce6120319e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45367 } } I0609 06:56:57.176735 2725 raft_consensus.cc:366] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:57.176919 2725 raft_consensus.cc:705] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9564879d3b7f4a318f5943ce6120319e, State: Initialized, Role: FOLLOWER I0609 06:56:57.177417 2725 consensus_queue.cc:229] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9564879d3b7f4a318f5943ce6120319e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45367 } } I0609 06:56:57.177824 2725 raft_consensus.cc:380] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:57.178030 2725 raft_consensus.cc:472] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:57.178230 2725 raft_consensus.cc:2890] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:57.182399 2725 raft_consensus.cc:494] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9564879d3b7f4a318f5943ce6120319e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45367 } } I0609 06:56:57.182871 2725 leader_election.cc:310] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9564879d3b7f4a318f5943ce6120319e; no voters: I0609 06:56:57.184132 2727 raft_consensus.cc:2644] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:57.184981 2725 leader_election.cc:296] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:57.186883 2727 raft_consensus.cc:667] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [term 1 LEADER]: Becoming Leader. State: Replica: 9564879d3b7f4a318f5943ce6120319e, State: Running, Role: LEADER I0609 06:56:57.187438 2725 ts_tablet_manager.cc:1082] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s I0609 06:56:57.187574 2727 consensus_queue.cc:206] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9564879d3b7f4a318f5943ce6120319e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45367 } } I0609 06:56:57.188062 2725 tablet_bootstrap.cc:439] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e: Bootstrap starting. I0609 06:56:57.191665 2725 tablet_bootstrap.cc:590] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e: No blocks or log segments found. Creating new log. I0609 06:56:57.193207 2605 catalog_manager.cc:3989] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e reported cstate change: term changed from 0 to 1, leader changed from to 9564879d3b7f4a318f5943ce6120319e (127.1.23.193). New cstate: current_term: 1 leader_uuid: "9564879d3b7f4a318f5943ce6120319e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9564879d3b7f4a318f5943ce6120319e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45367 } health_report { overall_health: HEALTHY } } } I0609 06:56:57.194790 2725 tablet_bootstrap.cc:439] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e: No bootstrap required, opened a new log I0609 06:56:57.195163 2725 ts_tablet_manager.cc:1054] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s I0609 06:56:57.196604 2725 raft_consensus.cc:340] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9564879d3b7f4a318f5943ce6120319e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45367 } } I0609 06:56:57.197120 2725 raft_consensus.cc:366] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:57.197376 2725 raft_consensus.cc:705] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9564879d3b7f4a318f5943ce6120319e, State: Initialized, Role: FOLLOWER I0609 06:56:57.197845 2725 consensus_queue.cc:229] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9564879d3b7f4a318f5943ce6120319e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45367 } } I0609 06:56:57.198338 2725 raft_consensus.cc:380] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:57.198532 2725 raft_consensus.cc:472] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:57.198837 2725 raft_consensus.cc:2890] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:57.203271 2725 raft_consensus.cc:494] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9564879d3b7f4a318f5943ce6120319e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45367 } } I0609 06:56:57.203790 2725 leader_election.cc:310] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9564879d3b7f4a318f5943ce6120319e; no voters: I0609 06:56:57.204255 2725 leader_election.cc:296] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:57.204391 2727 raft_consensus.cc:2644] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:57.204843 2727 raft_consensus.cc:667] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [term 1 LEADER]: Becoming Leader. State: Replica: 9564879d3b7f4a318f5943ce6120319e, State: Running, Role: LEADER I0609 06:56:57.205387 2727 consensus_queue.cc:206] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9564879d3b7f4a318f5943ce6120319e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45367 } } I0609 06:56:57.205796 2725 ts_tablet_manager.cc:1082] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e: Time spent starting tablet: real 0.010s user 0.009s sys 0.002s I0609 06:56:57.209491 2604 catalog_manager.cc:3989] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e reported cstate change: term changed from 0 to 1, leader changed from to 9564879d3b7f4a318f5943ce6120319e (127.1.23.193). New cstate: current_term: 1 leader_uuid: "9564879d3b7f4a318f5943ce6120319e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9564879d3b7f4a318f5943ce6120319e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45367 } health_report { overall_health: HEALTHY } } } I0609 06:56:57.560395 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:57.586671 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:57.587026 1119 tablet_replica.cc:265] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e: stopping tablet replica I0609 06:56:57.587685 1119 raft_consensus.cc:2147] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:57.588063 1119 raft_consensus.cc:2174] T 676c74517fd649188a865c12ae1d49c4 P 9564879d3b7f4a318f5943ce6120319e [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:57.591464 1119 tablet_replica.cc:265] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e: stopping tablet replica I0609 06:56:57.591753 1119 raft_consensus.cc:2147] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:57.591975 1119 raft_consensus.cc:2174] T 2450fc3e86c740899f80f39754045222 P 9564879d3b7f4a318f5943ce6120319e [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:57.611776 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:57.611945 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:57.612248 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:57.618535 1119 master.cc:247] Master@127.1.23.254:41285 shutting down... I0609 06:56:57.632007 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:57.632278 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:57.632505 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 9bd45aac38f34cce8c7c85a157a1a38c: stopping tablet replica I0609 06:56:57.640830 1119 master.cc:258] Master@127.1.23.254:41285 shutdown complete. [ OK ] ClientTest.TestMasterLookupPermits (870 ms) [ RUN ] ClientTest.TestCreateTableWithTooLongColumnName I0609 06:56:57.653574 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:41857 I0609 06:56:57.664880 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 95516us I0609 06:56:57.665153 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:57.665289 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:57.665485 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:57.665583 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:57.666360 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestCreateTableWithTooLongColumnName.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "4027be6a5c7c403eba03b774485df985" format_stamp: "Formatted at 2019-06-09 06:56:57 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:57.674389 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s I0609 06:56:57.675626 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:57.681826 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:57.681957 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestCreateTableWithTooLongColumnName.1560063404629499-1119/minicluster-data/master-0-root uuid: "4027be6a5c7c403eba03b774485df985" format_stamp: "Formatted at 2019-06-09 06:56:57 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:57.682145 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestCreateTableWithTooLongColumnName.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestCreateTableWithTooLongColumnName.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestCreateTableWithTooLongColumnName.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:57.773077 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:41857 I0609 06:56:57.774755 1119 webserver.cc:374] Webserver started at http://127.1.23.254:45413/ using document root and password file I0609 06:56:57.776252 2791 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:57.783663 2791 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985: Bootstrap starting. I0609 06:56:57.785699 2791 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985: No blocks or log segments found. Creating new log. I0609 06:56:57.788377 2791 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985: No bootstrap required, opened a new log I0609 06:56:57.789575 2791 raft_consensus.cc:340] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4027be6a5c7c403eba03b774485df985" member_type: VOTER } I0609 06:56:57.789891 2791 raft_consensus.cc:366] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:57.790122 2791 raft_consensus.cc:705] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4027be6a5c7c403eba03b774485df985, State: Initialized, Role: FOLLOWER I0609 06:56:57.790565 2791 consensus_queue.cc:229] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4027be6a5c7c403eba03b774485df985" member_type: VOTER } I0609 06:56:57.790926 2791 raft_consensus.cc:380] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:57.791106 2791 raft_consensus.cc:472] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:57.791352 2791 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:57.794481 2791 raft_consensus.cc:494] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4027be6a5c7c403eba03b774485df985" member_type: VOTER } I0609 06:56:57.794857 2791 leader_election.cc:310] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4027be6a5c7c403eba03b774485df985; no voters: I0609 06:56:57.796114 2791 leader_election.cc:296] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:57.796310 2793 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:57.797813 2793 raft_consensus.cc:667] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [term 1 LEADER]: Becoming Leader. State: Replica: 4027be6a5c7c403eba03b774485df985, State: Running, Role: LEADER I0609 06:56:57.798182 2793 consensus_queue.cc:206] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4027be6a5c7c403eba03b774485df985" member_type: VOTER } I0609 06:56:57.799959 2791 sys_catalog.cc:338] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "4027be6a5c7c403eba03b774485df985" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4027be6a5c7c403eba03b774485df985" member_type: VOTER } } I0609 06:56:57.800582 2791 sys_catalog.cc:341] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [sys.catalog]: This master's current role is: LEADER I0609 06:56:57.800415 2794 sys_catalog.cc:338] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4027be6a5c7c403eba03b774485df985" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4027be6a5c7c403eba03b774485df985" member_type: VOTER } } I0609 06:56:57.800667 2795 sys_catalog.cc:338] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4027be6a5c7c403eba03b774485df985. Latest consensus state: current_term: 1 leader_uuid: "4027be6a5c7c403eba03b774485df985" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4027be6a5c7c403eba03b774485df985" member_type: VOTER } } I0609 06:56:57.800979 2794 sys_catalog.cc:341] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [sys.catalog]: This master's current role is: LEADER I0609 06:56:57.801146 2795 sys_catalog.cc:341] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [sys.catalog]: This master's current role is: LEADER I0609 06:56:57.803582 2791 sys_catalog.cc:426] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:57.803731 2798 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:57.805711 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:57.806440 2798 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:57.839596 2798 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:57.841029 2798 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:57.849907 2798 catalog_manager.cc:4293] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985: Generated new TSK 0 I0609 06:56:57.884181 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 95516us I0609 06:56:57.884488 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:57.884616 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:57.884784 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:57.884888 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:57.885746 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestCreateTableWithTooLongColumnName.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" format_stamp: "Formatted at 2019-06-09 06:56:57 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:57.895256 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.009s user 0.009s sys 0.000s I0609 06:56:57.896540 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:57.903978 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:57.904120 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestCreateTableWithTooLongColumnName.1560063404629499-1119/minicluster-data/ts-0-root uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" format_stamp: "Formatted at 2019-06-09 06:56:57 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:57.904340 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestCreateTableWithTooLongColumnName.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestCreateTableWithTooLongColumnName.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestCreateTableWithTooLongColumnName.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:57.947679 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:57.947824 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:57.987004 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:38627 I0609 06:56:57.988644 1119 webserver.cc:374] Webserver started at http://127.1.23.193:43677/ using document root and password file I0609 06:56:57.999929 2865 heartbeater.cc:324] Connected to a master server at 127.1.23.254:41857 I0609 06:56:58.000133 2865 heartbeater.cc:405] Registering TS with master... I0609 06:56:58.000463 2865 heartbeater.cc:454] Master 127.1.23.254:41857 requested a full tablet report, sending... I0609 06:56:58.001868 2758 ts_manager.cc:161] Registered new tserver with Master: 1756ecfbc7ed43e38612ab7e20bb7f22 (127.1.23.193:38627) I0609 06:56:58.002861 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.012823073s I0609 06:56:58.004070 2758 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59006 I0609 06:56:58.016755 2865 heartbeater.cc:446] Master 127.1.23.254:41857 was elected leader, sending a full tablet report... I0609 06:56:58.021209 2758 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59008: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:58.044876 2832 tablet_service.cc:1016] Processing CreateTablet for tablet d3e8312276f847a09b1fb6fabe20a6a8 (table=client-testtb [id=737ab9c6f95a4b30b7d1234104ded82b]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:58.045814 2831 tablet_service.cc:1016] Processing CreateTablet for tablet f00841bb1ad742d6889363d59594dd4d (table=client-testtb [id=737ab9c6f95a4b30b7d1234104ded82b]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:58.046455 2831 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet f00841bb1ad742d6889363d59594dd4d. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:58.049666 2832 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet d3e8312276f847a09b1fb6fabe20a6a8. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:58.063716 2877 tablet_bootstrap.cc:439] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22: Bootstrap starting. I0609 06:56:58.065862 2877 tablet_bootstrap.cc:590] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22: No blocks or log segments found. Creating new log. I0609 06:56:58.068538 2877 tablet_bootstrap.cc:439] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22: No bootstrap required, opened a new log I0609 06:56:58.068876 2877 ts_tablet_manager.cc:1054] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s I0609 06:56:58.070214 2877 raft_consensus.cc:340] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 38627 } } I0609 06:56:58.070595 2877 raft_consensus.cc:366] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:58.070835 2877 raft_consensus.cc:705] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1756ecfbc7ed43e38612ab7e20bb7f22, State: Initialized, Role: FOLLOWER I0609 06:56:58.071213 2877 consensus_queue.cc:229] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 38627 } } I0609 06:56:58.071600 2877 raft_consensus.cc:380] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:58.071801 2877 raft_consensus.cc:472] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:58.072024 2877 raft_consensus.cc:2890] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:58.075220 2877 raft_consensus.cc:494] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 38627 } } I0609 06:56:58.075637 2877 leader_election.cc:310] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1756ecfbc7ed43e38612ab7e20bb7f22; no voters: I0609 06:56:58.076665 2877 leader_election.cc:296] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:58.076862 2879 raft_consensus.cc:2644] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:58.078582 2877 ts_tablet_manager.cc:1082] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s I0609 06:56:58.078071 2879 raft_consensus.cc:667] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 1 LEADER]: Becoming Leader. State: Replica: 1756ecfbc7ed43e38612ab7e20bb7f22, State: Running, Role: LEADER I0609 06:56:58.079145 2877 tablet_bootstrap.cc:439] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22: Bootstrap starting. I0609 06:56:58.079234 2879 consensus_queue.cc:206] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 38627 } } I0609 06:56:58.081853 2877 tablet_bootstrap.cc:590] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22: No blocks or log segments found. Creating new log. I0609 06:56:58.083169 2757 catalog_manager.cc:3989] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 reported cstate change: term changed from 0 to 1, leader changed from to 1756ecfbc7ed43e38612ab7e20bb7f22 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 38627 } health_report { overall_health: HEALTHY } } } I0609 06:56:58.085837 2877 tablet_bootstrap.cc:439] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22: No bootstrap required, opened a new log I0609 06:56:58.086110 2877 ts_tablet_manager.cc:1054] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s I0609 06:56:58.087448 2877 raft_consensus.cc:340] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 38627 } } I0609 06:56:58.087925 2877 raft_consensus.cc:366] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:58.088150 2877 raft_consensus.cc:705] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1756ecfbc7ed43e38612ab7e20bb7f22, State: Initialized, Role: FOLLOWER I0609 06:56:58.088675 2877 consensus_queue.cc:229] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 38627 } } I0609 06:56:58.089390 2877 raft_consensus.cc:380] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:58.089706 2877 raft_consensus.cc:472] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:58.090049 2877 raft_consensus.cc:2890] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:58.094547 2877 raft_consensus.cc:494] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 38627 } } I0609 06:56:58.094966 2877 leader_election.cc:310] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1756ecfbc7ed43e38612ab7e20bb7f22; no voters: I0609 06:56:58.095418 2877 leader_election.cc:296] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:58.095538 2879 raft_consensus.cc:2644] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:58.096096 2879 raft_consensus.cc:667] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 1 LEADER]: Becoming Leader. State: Replica: 1756ecfbc7ed43e38612ab7e20bb7f22, State: Running, Role: LEADER I0609 06:56:58.096343 2877 ts_tablet_manager.cc:1082] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s I0609 06:56:58.096567 2879 consensus_queue.cc:206] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 38627 } } I0609 06:56:58.100111 2758 catalog_manager.cc:3989] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 reported cstate change: term changed from 0 to 1, leader changed from to 1756ecfbc7ed43e38612ab7e20bb7f22 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756ecfbc7ed43e38612ab7e20bb7f22" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 38627 } health_report { overall_health: HEALTHY } } } I0609 06:56:58.109942 2758 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59008: name: "foobar" schema { columns { name: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } split_rows_range_bounds { } partition_schema { range_schema { columns { name: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" } } } I0609 06:56:58.115039 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:58.134083 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:58.134455 1119 tablet_replica.cc:265] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22: stopping tablet replica I0609 06:56:58.134793 1119 raft_consensus.cc:2147] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:58.135054 1119 raft_consensus.cc:2174] T f00841bb1ad742d6889363d59594dd4d P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:58.137403 1119 tablet_replica.cc:265] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22: stopping tablet replica I0609 06:56:58.137684 1119 raft_consensus.cc:2147] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:58.137935 1119 raft_consensus.cc:2174] T d3e8312276f847a09b1fb6fabe20a6a8 P 1756ecfbc7ed43e38612ab7e20bb7f22 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:58.149479 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:58.149608 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:58.149852 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:58.155690 1119 master.cc:247] Master@127.1.23.254:41857 shutting down... I0609 06:56:58.170565 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:58.170822 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:58.171118 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 4027be6a5c7c403eba03b774485df985: stopping tablet replica I0609 06:56:58.182482 1119 master.cc:258] Master@127.1.23.254:41857 shutdown complete. [ OK ] ClientTest.TestCreateTableWithTooLongColumnName (542 ms) [ RUN ] ClientTest.TestClonePredicates I0609 06:56:58.196018 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:44631 I0609 06:56:58.208144 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 96016us I0609 06:56:58.208425 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:58.208544 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:58.208709 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:58.208811 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:58.209686 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestClonePredicates.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "8e44df63d6fa488a9ae3732910781026" format_stamp: "Formatted at 2019-06-09 06:56:58 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:58.217471 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s I0609 06:56:58.218685 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:58.224908 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s I0609 06:56:58.225049 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestClonePredicates.1560063404629499-1119/minicluster-data/master-0-root uuid: "8e44df63d6fa488a9ae3732910781026" format_stamp: "Formatted at 2019-06-09 06:56:58 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:58.225244 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestClonePredicates.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestClonePredicates.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestClonePredicates.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:58.285387 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:44631 I0609 06:56:58.286980 1119 webserver.cc:374] Webserver started at http://127.1.23.254:36373/ using document root and password file I0609 06:56:58.288441 2940 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:58.295434 2940 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026: Bootstrap starting. I0609 06:56:58.297343 2940 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026: No blocks or log segments found. Creating new log. I0609 06:56:58.299811 2940 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026: No bootstrap required, opened a new log I0609 06:56:58.300900 2940 raft_consensus.cc:340] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e44df63d6fa488a9ae3732910781026" member_type: VOTER } I0609 06:56:58.301187 2940 raft_consensus.cc:366] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:58.301360 2940 raft_consensus.cc:705] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e44df63d6fa488a9ae3732910781026, State: Initialized, Role: FOLLOWER I0609 06:56:58.301741 2940 consensus_queue.cc:229] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e44df63d6fa488a9ae3732910781026" member_type: VOTER } I0609 06:56:58.302098 2940 raft_consensus.cc:380] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:58.302263 2940 raft_consensus.cc:472] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:58.302458 2940 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:58.305513 2940 raft_consensus.cc:494] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e44df63d6fa488a9ae3732910781026" member_type: VOTER } I0609 06:56:58.305884 2940 leader_election.cc:310] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e44df63d6fa488a9ae3732910781026; no voters: I0609 06:56:58.306990 2942 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:58.307603 2940 leader_election.cc:296] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:58.308727 2942 raft_consensus.cc:667] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [term 1 LEADER]: Becoming Leader. State: Replica: 8e44df63d6fa488a9ae3732910781026, State: Running, Role: LEADER I0609 06:56:58.309192 2942 consensus_queue.cc:206] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e44df63d6fa488a9ae3732910781026" member_type: VOTER } I0609 06:56:58.310959 2940 sys_catalog.cc:338] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "8e44df63d6fa488a9ae3732910781026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e44df63d6fa488a9ae3732910781026" member_type: VOTER } } I0609 06:56:58.311121 2944 sys_catalog.cc:338] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e44df63d6fa488a9ae3732910781026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e44df63d6fa488a9ae3732910781026" member_type: VOTER } } I0609 06:56:58.311312 2943 sys_catalog.cc:338] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e44df63d6fa488a9ae3732910781026. Latest consensus state: current_term: 1 leader_uuid: "8e44df63d6fa488a9ae3732910781026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e44df63d6fa488a9ae3732910781026" member_type: VOTER } } I0609 06:56:58.311591 2940 sys_catalog.cc:341] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [sys.catalog]: This master's current role is: LEADER I0609 06:56:58.311710 2944 sys_catalog.cc:341] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [sys.catalog]: This master's current role is: LEADER I0609 06:56:58.311870 2943 sys_catalog.cc:341] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [sys.catalog]: This master's current role is: LEADER I0609 06:56:58.314172 2940 sys_catalog.cc:426] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:58.314338 2946 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:58.316670 2946 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:58.317742 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:58.404171 2946 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:58.405563 2946 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:58.413074 2946 catalog_manager.cc:4293] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026: Generated new TSK 0 I0609 06:56:58.459614 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 96016us I0609 06:56:58.459966 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:58.460132 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:58.460386 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:58.460500 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:58.461468 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestClonePredicates.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "4c292794ac27412eafb8e03355749e2e" format_stamp: "Formatted at 2019-06-09 06:56:58 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:58.470122 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.006s sys 0.002s I0609 06:56:58.471357 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:58.478816 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:58.478957 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestClonePredicates.1560063404629499-1119/minicluster-data/ts-0-root uuid: "4c292794ac27412eafb8e03355749e2e" format_stamp: "Formatted at 2019-06-09 06:56:58 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:58.479187 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestClonePredicates.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestClonePredicates.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestClonePredicates.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:58.526067 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:58.526232 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:58.565682 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:41293 I0609 06:56:58.567351 1119 webserver.cc:374] Webserver started at http://127.1.23.193:35345/ using document root and password file I0609 06:56:58.578864 3014 heartbeater.cc:324] Connected to a master server at 127.1.23.254:44631 I0609 06:56:58.579138 3014 heartbeater.cc:405] Registering TS with master... I0609 06:56:58.579532 3014 heartbeater.cc:454] Master 127.1.23.254:44631 requested a full tablet report, sending... I0609 06:56:58.580983 2907 ts_manager.cc:161] Registered new tserver with Master: 4c292794ac27412eafb8e03355749e2e (127.1.23.193:41293) I0609 06:56:58.581538 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.012775897s I0609 06:56:58.583578 2907 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35426 I0609 06:56:58.596333 3014 heartbeater.cc:446] Master 127.1.23.254:44631 was elected leader, sending a full tablet report... I0609 06:56:58.600081 2906 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35428: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:58.623809 2981 tablet_service.cc:1016] Processing CreateTablet for tablet c9b7a4f1387045c18a8f94d6e07ea77d (table=client-testtb [id=e481014a611c41a8b2166cb55a77910b]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:58.624120 2980 tablet_service.cc:1016] Processing CreateTablet for tablet fcdd3529526647de93143c70e55bcb56 (table=client-testtb [id=e481014a611c41a8b2166cb55a77910b]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:58.624568 2981 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet c9b7a4f1387045c18a8f94d6e07ea77d. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:58.625027 2980 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet fcdd3529526647de93143c70e55bcb56. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:58.633764 3026 tablet_bootstrap.cc:439] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e: Bootstrap starting. I0609 06:56:58.636875 3026 tablet_bootstrap.cc:590] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e: No blocks or log segments found. Creating new log. I0609 06:56:58.639770 3026 tablet_bootstrap.cc:439] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e: No bootstrap required, opened a new log I0609 06:56:58.640056 3026 ts_tablet_manager.cc:1054] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s I0609 06:56:58.641218 3026 raft_consensus.cc:340] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c292794ac27412eafb8e03355749e2e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41293 } } I0609 06:56:58.641621 3026 raft_consensus.cc:366] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:58.641901 3026 raft_consensus.cc:705] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c292794ac27412eafb8e03355749e2e, State: Initialized, Role: FOLLOWER I0609 06:56:58.642400 3026 consensus_queue.cc:229] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c292794ac27412eafb8e03355749e2e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41293 } } I0609 06:56:58.642907 3026 raft_consensus.cc:380] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:58.643201 3026 raft_consensus.cc:472] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:58.643434 3026 raft_consensus.cc:2890] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:58.647119 3026 raft_consensus.cc:494] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c292794ac27412eafb8e03355749e2e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41293 } } I0609 06:56:58.647753 3026 leader_election.cc:310] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c292794ac27412eafb8e03355749e2e; no voters: I0609 06:56:58.649143 3026 leader_election.cc:296] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:58.649375 3028 raft_consensus.cc:2644] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:58.650921 3028 raft_consensus.cc:667] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [term 1 LEADER]: Becoming Leader. State: Replica: 4c292794ac27412eafb8e03355749e2e, State: Running, Role: LEADER I0609 06:56:58.652185 3026 ts_tablet_manager.cc:1082] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s I0609 06:56:58.652140 3028 consensus_queue.cc:206] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c292794ac27412eafb8e03355749e2e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41293 } } I0609 06:56:58.652700 3026 tablet_bootstrap.cc:439] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e: Bootstrap starting. I0609 06:56:58.654615 3026 tablet_bootstrap.cc:590] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e: No blocks or log segments found. Creating new log. I0609 06:56:58.657975 3026 tablet_bootstrap.cc:439] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e: No bootstrap required, opened a new log I0609 06:56:58.658375 3026 ts_tablet_manager.cc:1054] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s I0609 06:56:58.658939 2906 catalog_manager.cc:3989] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e reported cstate change: term changed from 0 to 1, leader changed from to 4c292794ac27412eafb8e03355749e2e (127.1.23.193). New cstate: current_term: 1 leader_uuid: "4c292794ac27412eafb8e03355749e2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c292794ac27412eafb8e03355749e2e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41293 } health_report { overall_health: HEALTHY } } } I0609 06:56:58.659598 3026 raft_consensus.cc:340] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c292794ac27412eafb8e03355749e2e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41293 } } I0609 06:56:58.660166 3026 raft_consensus.cc:366] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:58.660475 3026 raft_consensus.cc:705] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c292794ac27412eafb8e03355749e2e, State: Initialized, Role: FOLLOWER I0609 06:56:58.661170 3026 consensus_queue.cc:229] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c292794ac27412eafb8e03355749e2e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41293 } } I0609 06:56:58.661722 3026 raft_consensus.cc:380] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:58.661919 3026 raft_consensus.cc:472] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:58.662145 3026 raft_consensus.cc:2890] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:58.666700 3026 raft_consensus.cc:494] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c292794ac27412eafb8e03355749e2e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41293 } } I0609 06:56:58.667109 3026 leader_election.cc:310] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c292794ac27412eafb8e03355749e2e; no voters: I0609 06:56:58.667526 3026 leader_election.cc:296] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:58.667659 3028 raft_consensus.cc:2644] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:58.668145 3028 raft_consensus.cc:667] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [term 1 LEADER]: Becoming Leader. State: Replica: 4c292794ac27412eafb8e03355749e2e, State: Running, Role: LEADER I0609 06:56:58.668382 3026 ts_tablet_manager.cc:1082] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s I0609 06:56:58.668656 3028 consensus_queue.cc:206] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c292794ac27412eafb8e03355749e2e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41293 } } I0609 06:56:58.672246 2906 catalog_manager.cc:3989] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e reported cstate change: term changed from 0 to 1, leader changed from to 4c292794ac27412eafb8e03355749e2e (127.1.23.193). New cstate: current_term: 1 leader_uuid: "4c292794ac27412eafb8e03355749e2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c292794ac27412eafb8e03355749e2e" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 41293 } health_report { overall_health: HEALTHY } } } I0609 06:56:58.719368 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:58.742995 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:58.743333 1119 tablet_replica.cc:265] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e: stopping tablet replica I0609 06:56:58.743748 1119 raft_consensus.cc:2147] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:58.744047 1119 raft_consensus.cc:2174] T fcdd3529526647de93143c70e55bcb56 P 4c292794ac27412eafb8e03355749e2e [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:58.746412 1119 tablet_replica.cc:265] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e: stopping tablet replica I0609 06:56:58.746747 1119 raft_consensus.cc:2147] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:58.746989 1119 raft_consensus.cc:2174] T c9b7a4f1387045c18a8f94d6e07ea77d P 4c292794ac27412eafb8e03355749e2e [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:58.766547 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:58.766665 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:58.766942 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:58.773367 1119 master.cc:247] Master@127.1.23.254:44631 shutting down... I0609 06:56:58.786013 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:58.786298 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:58.786572 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 8e44df63d6fa488a9ae3732910781026: stopping tablet replica I0609 06:56:58.804744 1119 master.cc:258] Master@127.1.23.254:44631 shutdown complete. [ OK ] ClientTest.TestClonePredicates (622 ms) [ RUN ] ClientTest.TestErrorCollector I0609 06:56:58.818549 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:39133 I0609 06:56:58.830042 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 96016us I0609 06:56:58.830313 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:58.830451 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:58.830622 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:58.830744 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:58.831527 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestErrorCollector.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "027af72dd998430c8b0e30ffb2e47579" format_stamp: "Formatted at 2019-06-09 06:56:58 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:58.839850 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s I0609 06:56:58.841099 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:58.847709 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s I0609 06:56:58.847858 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestErrorCollector.1560063404629499-1119/minicluster-data/master-0-root uuid: "027af72dd998430c8b0e30ffb2e47579" format_stamp: "Formatted at 2019-06-09 06:56:58 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:58.848052 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestErrorCollector.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestErrorCollector.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestErrorCollector.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:58.899291 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:39133 I0609 06:56:58.900794 1119 webserver.cc:374] Webserver started at http://127.1.23.254:43959/ using document root and password file I0609 06:56:58.902328 3091 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:58.909633 3091 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579: Bootstrap starting. I0609 06:56:58.911656 3091 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579: No blocks or log segments found. Creating new log. I0609 06:56:58.914229 3091 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579: No bootstrap required, opened a new log I0609 06:56:58.915398 3091 raft_consensus.cc:340] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "027af72dd998430c8b0e30ffb2e47579" member_type: VOTER } I0609 06:56:58.915724 3091 raft_consensus.cc:366] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:58.915874 3091 raft_consensus.cc:705] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 027af72dd998430c8b0e30ffb2e47579, State: Initialized, Role: FOLLOWER I0609 06:56:58.916256 3091 consensus_queue.cc:229] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "027af72dd998430c8b0e30ffb2e47579" member_type: VOTER } I0609 06:56:58.916606 3091 raft_consensus.cc:380] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:58.916779 3091 raft_consensus.cc:472] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:58.916975 3091 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:58.920712 3091 raft_consensus.cc:494] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "027af72dd998430c8b0e30ffb2e47579" member_type: VOTER } I0609 06:56:58.921113 3091 leader_election.cc:310] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 027af72dd998430c8b0e30ffb2e47579; no voters: I0609 06:56:58.922379 3093 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:58.923015 3091 leader_election.cc:296] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:58.924110 3093 raft_consensus.cc:667] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [term 1 LEADER]: Becoming Leader. State: Replica: 027af72dd998430c8b0e30ffb2e47579, State: Running, Role: LEADER I0609 06:56:58.924537 3093 consensus_queue.cc:206] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "027af72dd998430c8b0e30ffb2e47579" member_type: VOTER } I0609 06:56:58.927333 3095 sys_catalog.cc:338] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 027af72dd998430c8b0e30ffb2e47579. Latest consensus state: current_term: 1 leader_uuid: "027af72dd998430c8b0e30ffb2e47579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "027af72dd998430c8b0e30ffb2e47579" member_type: VOTER } } I0609 06:56:58.927700 3095 sys_catalog.cc:341] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [sys.catalog]: This master's current role is: LEADER I0609 06:56:58.928963 3091 sys_catalog.cc:338] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "027af72dd998430c8b0e30ffb2e47579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "027af72dd998430c8b0e30ffb2e47579" member_type: VOTER } } I0609 06:56:58.929538 3091 sys_catalog.cc:341] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [sys.catalog]: This master's current role is: LEADER I0609 06:56:58.930598 3091 sys_catalog.cc:426] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:58.930845 3099 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:58.931114 3094 sys_catalog.cc:338] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "027af72dd998430c8b0e30ffb2e47579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "027af72dd998430c8b0e30ffb2e47579" member_type: VOTER } } I0609 06:56:58.931788 3094 sys_catalog.cc:341] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [sys.catalog]: This master's current role is: LEADER I0609 06:56:58.933732 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:58.934111 3099 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:58.972447 3099 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:58.973939 3099 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:58.986075 3099 catalog_manager.cc:4293] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579: Generated new TSK 0 I0609 06:56:59.011247 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 96516us I0609 06:56:59.011536 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:59.011665 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:59.011849 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:59.011958 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:59.012773 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestErrorCollector.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "0c0ecb57c443435d84a1bd6987c3305d" format_stamp: "Formatted at 2019-06-09 06:56:59 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:59.021826 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.009s user 0.010s sys 0.001s I0609 06:56:59.023089 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:59.029976 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s I0609 06:56:59.030117 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestErrorCollector.1560063404629499-1119/minicluster-data/ts-0-root uuid: "0c0ecb57c443435d84a1bd6987c3305d" format_stamp: "Formatted at 2019-06-09 06:56:59 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:59.030330 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestErrorCollector.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestErrorCollector.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestErrorCollector.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:59.055289 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:59.055419 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:59.091054 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:46675 I0609 06:56:59.092839 1119 webserver.cc:374] Webserver started at http://127.1.23.193:39437/ using document root and password file I0609 06:56:59.104132 3166 heartbeater.cc:324] Connected to a master server at 127.1.23.254:39133 I0609 06:56:59.104442 3166 heartbeater.cc:405] Registering TS with master... I0609 06:56:59.104856 3166 heartbeater.cc:454] Master 127.1.23.254:39133 requested a full tablet report, sending... I0609 06:56:59.106215 3058 ts_manager.cc:161] Registered new tserver with Master: 0c0ecb57c443435d84a1bd6987c3305d (127.1.23.193:46675) I0609 06:56:59.106562 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.011689078s I0609 06:56:59.108060 3058 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40750 I0609 06:56:59.120859 3166 heartbeater.cc:446] Master 127.1.23.254:39133 was elected leader, sending a full tablet report... I0609 06:56:59.128469 3058 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40752: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:59.157692 3133 tablet_service.cc:1016] Processing CreateTablet for tablet 026f9bffd11c4a39a04a0797228c34d2 (table=client-testtb [id=ade01509451a498f97f98913a4ad64cb]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:59.158072 3132 tablet_service.cc:1016] Processing CreateTablet for tablet f78343c1a1d04d86a483cb9a3476c5c9 (table=client-testtb [id=ade01509451a498f97f98913a4ad64cb]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:59.158609 3133 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 026f9bffd11c4a39a04a0797228c34d2. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:59.159312 3132 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet f78343c1a1d04d86a483cb9a3476c5c9. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:59.168402 3178 tablet_bootstrap.cc:439] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d: Bootstrap starting. I0609 06:56:59.170581 3178 tablet_bootstrap.cc:590] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d: No blocks or log segments found. Creating new log. I0609 06:56:59.173633 3178 tablet_bootstrap.cc:439] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d: No bootstrap required, opened a new log I0609 06:56:59.174013 3178 ts_tablet_manager.cc:1054] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s I0609 06:56:59.175456 3178 raft_consensus.cc:340] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0ecb57c443435d84a1bd6987c3305d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46675 } } I0609 06:56:59.175817 3178 raft_consensus.cc:366] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:59.175992 3178 raft_consensus.cc:705] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0ecb57c443435d84a1bd6987c3305d, State: Initialized, Role: FOLLOWER I0609 06:56:59.176409 3178 consensus_queue.cc:229] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0ecb57c443435d84a1bd6987c3305d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46675 } } I0609 06:56:59.176801 3178 raft_consensus.cc:380] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:59.176990 3178 raft_consensus.cc:472] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:59.177215 3178 raft_consensus.cc:2890] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:59.181025 3178 raft_consensus.cc:494] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0ecb57c443435d84a1bd6987c3305d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46675 } } I0609 06:56:59.181507 3178 leader_election.cc:310] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c0ecb57c443435d84a1bd6987c3305d; no voters: I0609 06:56:59.182646 3180 raft_consensus.cc:2644] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:59.183324 3178 leader_election.cc:296] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:59.184839 3180 raft_consensus.cc:667] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [term 1 LEADER]: Becoming Leader. State: Replica: 0c0ecb57c443435d84a1bd6987c3305d, State: Running, Role: LEADER I0609 06:56:59.185252 3178 ts_tablet_manager.cc:1082] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d: Time spent starting tablet: real 0.011s user 0.006s sys 0.003s I0609 06:56:59.185379 3180 consensus_queue.cc:206] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0ecb57c443435d84a1bd6987c3305d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46675 } } I0609 06:56:59.185932 3178 tablet_bootstrap.cc:439] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d: Bootstrap starting. I0609 06:56:59.189357 3178 tablet_bootstrap.cc:590] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d: No blocks or log segments found. Creating new log. I0609 06:56:59.190333 3058 catalog_manager.cc:3989] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d reported cstate change: term changed from 0 to 1, leader changed from to 0c0ecb57c443435d84a1bd6987c3305d (127.1.23.193). New cstate: current_term: 1 leader_uuid: "0c0ecb57c443435d84a1bd6987c3305d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0ecb57c443435d84a1bd6987c3305d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46675 } health_report { overall_health: HEALTHY } } } I0609 06:56:59.193141 3178 tablet_bootstrap.cc:439] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d: No bootstrap required, opened a new log I0609 06:56:59.193456 3178 ts_tablet_manager.cc:1054] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s I0609 06:56:59.194665 3178 raft_consensus.cc:340] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0ecb57c443435d84a1bd6987c3305d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46675 } } I0609 06:56:59.195323 3178 raft_consensus.cc:366] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:59.195546 3178 raft_consensus.cc:705] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0ecb57c443435d84a1bd6987c3305d, State: Initialized, Role: FOLLOWER I0609 06:56:59.196032 3178 consensus_queue.cc:229] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0ecb57c443435d84a1bd6987c3305d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46675 } } I0609 06:56:59.196606 3178 raft_consensus.cc:380] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:59.196799 3178 raft_consensus.cc:472] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:59.197016 3178 raft_consensus.cc:2890] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:59.202039 3178 raft_consensus.cc:494] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0ecb57c443435d84a1bd6987c3305d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46675 } } I0609 06:56:59.202539 3178 leader_election.cc:310] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c0ecb57c443435d84a1bd6987c3305d; no voters: I0609 06:56:59.203164 3178 leader_election.cc:296] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:59.203254 3180 raft_consensus.cc:2644] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:59.203861 3180 raft_consensus.cc:667] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [term 1 LEADER]: Becoming Leader. State: Replica: 0c0ecb57c443435d84a1bd6987c3305d, State: Running, Role: LEADER I0609 06:56:59.204125 3178 ts_tablet_manager.cc:1082] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s I0609 06:56:59.204198 3180 consensus_queue.cc:206] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0ecb57c443435d84a1bd6987c3305d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46675 } } I0609 06:56:59.207933 3058 catalog_manager.cc:3989] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d reported cstate change: term changed from 0 to 1, leader changed from to 0c0ecb57c443435d84a1bd6987c3305d (127.1.23.193). New cstate: current_term: 1 leader_uuid: "0c0ecb57c443435d84a1bd6987c3305d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0ecb57c443435d84a1bd6987c3305d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 46675 } health_report { overall_health: HEALTHY } } } I0609 06:56:59.245196 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:59.265450 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:59.265794 1119 tablet_replica.cc:265] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d: stopping tablet replica I0609 06:56:59.266151 1119 raft_consensus.cc:2147] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:59.266446 1119 raft_consensus.cc:2174] T f78343c1a1d04d86a483cb9a3476c5c9 P 0c0ecb57c443435d84a1bd6987c3305d [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:59.268584 1119 tablet_replica.cc:265] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d: stopping tablet replica I0609 06:56:59.268872 1119 raft_consensus.cc:2147] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:59.269105 1119 raft_consensus.cc:2174] T 026f9bffd11c4a39a04a0797228c34d2 P 0c0ecb57c443435d84a1bd6987c3305d [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:59.288609 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:59.288745 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:59.289026 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:59.295050 1119 master.cc:247] Master@127.1.23.254:39133 shutting down... I0609 06:56:59.308264 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:59.308504 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:59.308861 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 027af72dd998430c8b0e30ffb2e47579: stopping tablet replica I0609 06:56:59.327888 1119 master.cc:258] Master@127.1.23.254:39133 shutdown complete. [ OK ] ClientTest.TestErrorCollector (523 ms) [ RUN ] ClientTest.TestBlockScannerHijackingAttempts I0609 06:56:59.341583 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:46755 I0609 06:56:59.353687 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 96516us I0609 06:56:59.353976 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:59.354125 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:59.354346 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:59.354456 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:59.355307 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "79487940486d4430a7133dacf260d00f" format_stamp: "Formatted at 2019-06-09 06:56:59 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:59.363189 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s I0609 06:56:59.364430 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:59.370892 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:59.371023 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/master-0-root uuid: "79487940486d4430a7133dacf260d00f" format_stamp: "Formatted at 2019-06-09 06:56:59 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:59.371323 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:59.418606 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:46755 I0609 06:56:59.420296 1119 webserver.cc:374] Webserver started at http://127.1.23.254:35561/ using document root and password file I0609 06:56:59.421831 3243 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:59.428827 3243 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f: Bootstrap starting. I0609 06:56:59.430924 3243 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f: No blocks or log segments found. Creating new log. I0609 06:56:59.433516 3243 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f: No bootstrap required, opened a new log I0609 06:56:59.434628 3243 raft_consensus.cc:340] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } I0609 06:56:59.434921 3243 raft_consensus.cc:366] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:59.435096 3243 raft_consensus.cc:705] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79487940486d4430a7133dacf260d00f, State: Initialized, Role: FOLLOWER I0609 06:56:59.435509 3243 consensus_queue.cc:229] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } I0609 06:56:59.435920 3243 raft_consensus.cc:380] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:59.436087 3243 raft_consensus.cc:472] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:59.436305 3243 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:59.439355 3243 raft_consensus.cc:494] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } I0609 06:56:59.439750 3243 leader_election.cc:310] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79487940486d4430a7133dacf260d00f; no voters: I0609 06:56:59.440935 3243 leader_election.cc:296] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:59.441125 3245 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:59.442764 3245 raft_consensus.cc:667] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 1 LEADER]: Becoming Leader. State: Replica: 79487940486d4430a7133dacf260d00f, State: Running, Role: LEADER I0609 06:56:59.443125 3245 consensus_queue.cc:206] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } I0609 06:56:59.445327 3243 sys_catalog.cc:338] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "79487940486d4430a7133dacf260d00f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } } I0609 06:56:59.445524 3246 sys_catalog.cc:338] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79487940486d4430a7133dacf260d00f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } } I0609 06:56:59.445982 3243 sys_catalog.cc:341] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: This master's current role is: LEADER I0609 06:56:59.446271 3246 sys_catalog.cc:341] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: This master's current role is: LEADER I0609 06:56:59.445521 3247 sys_catalog.cc:338] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79487940486d4430a7133dacf260d00f. Latest consensus state: current_term: 1 leader_uuid: "79487940486d4430a7133dacf260d00f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } } I0609 06:56:59.446799 3247 sys_catalog.cc:341] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: This master's current role is: LEADER I0609 06:56:59.448448 3243 sys_catalog.cc:426] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:59.448619 3250 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:59.449702 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:59.451234 3250 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:59.495368 3250 catalog_manager.cc:951] Generated new certificate authority record I0609 06:56:59.496790 3250 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:59.508566 3250 catalog_manager.cc:4293] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f: Generated new TSK 0 I0609 06:56:59.527501 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 96516us I0609 06:56:59.527822 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:59.527989 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:56:59.528228 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:56:59.528381 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:56:59.529250 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" format_stamp: "Formatted at 2019-06-09 06:56:59 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:59.537824 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.008s sys 0.000s I0609 06:56:59.539150 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:59.546015 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:59.546149 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/ts-0-root uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" format_stamp: "Formatted at 2019-06-09 06:56:59 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:59.546361 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:59.572247 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:56:59.572403 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:56:59.612527 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:45363 I0609 06:56:59.614284 1119 webserver.cc:374] Webserver started at http://127.1.23.193:39525/ using document root and password file I0609 06:56:59.626245 3317 heartbeater.cc:324] Connected to a master server at 127.1.23.254:46755 I0609 06:56:59.626480 3317 heartbeater.cc:405] Registering TS with master... I0609 06:56:59.626871 3317 heartbeater.cc:454] Master 127.1.23.254:46755 requested a full tablet report, sending... I0609 06:56:59.628266 3210 ts_manager.cc:161] Registered new tserver with Master: 37c1ed11cd8c4d7287c87baa7cb3db4d (127.1.23.193:45363) I0609 06:56:59.628707 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.012523759s I0609 06:56:59.630039 3210 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45092 I0609 06:56:59.642577 3317 heartbeater.cc:446] Master 127.1.23.254:46755 was elected leader, sending a full tablet report... I0609 06:56:59.645906 3210 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45094: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:56:59.674556 3284 tablet_service.cc:1016] Processing CreateTablet for tablet 49ea677949a8499480e011d8481e2c6a (table=client-testtb [id=cf0e3acd9a4b4441aeee6a97086eaffe]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:56:59.675083 3283 tablet_service.cc:1016] Processing CreateTablet for tablet 5964356d13164391b488b81a09e4f9b0 (table=client-testtb [id=cf0e3acd9a4b4441aeee6a97086eaffe]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:56:59.675489 3284 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 49ea677949a8499480e011d8481e2c6a. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:59.675942 3283 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 5964356d13164391b488b81a09e4f9b0. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:56:59.690999 3329 tablet_bootstrap.cc:439] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d: Bootstrap starting. I0609 06:56:59.693431 3329 tablet_bootstrap.cc:590] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d: No blocks or log segments found. Creating new log. I0609 06:56:59.696082 3329 tablet_bootstrap.cc:439] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d: No bootstrap required, opened a new log I0609 06:56:59.696403 3329 ts_tablet_manager.cc:1054] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s I0609 06:56:59.697820 3329 raft_consensus.cc:340] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:56:59.698197 3329 raft_consensus.cc:366] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:59.698388 3329 raft_consensus.cc:705] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37c1ed11cd8c4d7287c87baa7cb3db4d, State: Initialized, Role: FOLLOWER I0609 06:56:59.698859 3329 consensus_queue.cc:229] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:56:59.699307 3329 raft_consensus.cc:380] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:59.699486 3329 raft_consensus.cc:472] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:59.699717 3329 raft_consensus.cc:2890] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:59.703169 3329 raft_consensus.cc:494] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:56:59.703635 3329 leader_election.cc:310] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37c1ed11cd8c4d7287c87baa7cb3db4d; no voters: I0609 06:56:59.705096 3329 leader_election.cc:296] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:59.705364 3331 raft_consensus.cc:2644] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:59.707425 3331 raft_consensus.cc:667] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 LEADER]: Becoming Leader. State: Replica: 37c1ed11cd8c4d7287c87baa7cb3db4d, State: Running, Role: LEADER I0609 06:56:59.708014 3329 ts_tablet_manager.cc:1082] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s I0609 06:56:59.707973 3331 consensus_queue.cc:206] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:56:59.708570 3329 tablet_bootstrap.cc:439] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d: Bootstrap starting. I0609 06:56:59.711802 3329 tablet_bootstrap.cc:590] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d: No blocks or log segments found. Creating new log. I0609 06:56:59.711578 3209 catalog_manager.cc:3989] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d reported cstate change: term changed from 0 to 1, leader changed from to 37c1ed11cd8c4d7287c87baa7cb3db4d (127.1.23.193). New cstate: current_term: 1 leader_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } health_report { overall_health: HEALTHY } } } I0609 06:56:59.715318 3329 tablet_bootstrap.cc:439] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d: No bootstrap required, opened a new log I0609 06:56:59.715708 3329 ts_tablet_manager.cc:1054] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s I0609 06:56:59.716917 3329 raft_consensus.cc:340] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:56:59.717350 3329 raft_consensus.cc:366] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:56:59.717624 3329 raft_consensus.cc:705] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37c1ed11cd8c4d7287c87baa7cb3db4d, State: Initialized, Role: FOLLOWER I0609 06:56:59.718116 3329 consensus_queue.cc:229] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:56:59.718830 3329 raft_consensus.cc:380] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:59.719027 3329 raft_consensus.cc:472] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:59.719305 3329 raft_consensus.cc:2890] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 0 FOLLOWER]: Advancing to term 1 I0609 06:56:59.723495 3329 raft_consensus.cc:494] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:56:59.723917 3329 leader_election.cc:310] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37c1ed11cd8c4d7287c87baa7cb3db4d; no voters: I0609 06:56:59.724356 3329 leader_election.cc:296] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:56:59.724489 3331 raft_consensus.cc:2644] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:56:59.724903 3331 raft_consensus.cc:667] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 LEADER]: Becoming Leader. State: Replica: 37c1ed11cd8c4d7287c87baa7cb3db4d, State: Running, Role: LEADER I0609 06:56:59.725416 3329 ts_tablet_manager.cc:1082] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d: Time spent starting tablet: real 0.009s user 0.009s sys 0.002s I0609 06:56:59.725486 3331 consensus_queue.cc:206] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:56:59.728930 3210 catalog_manager.cc:3989] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d reported cstate change: term changed from 0 to 1, leader changed from to 37c1ed11cd8c4d7287c87baa7cb3db4d (127.1.23.193). New cstate: current_term: 1 leader_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } health_report { overall_health: HEALTHY } } } I0609 06:56:59.752133 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:59.769203 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:56:59.769570 1119 tablet_replica.cc:265] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d: stopping tablet replica I0609 06:56:59.769920 1119 raft_consensus.cc:2147] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:59.770205 1119 raft_consensus.cc:2174] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:59.772630 1119 tablet_replica.cc:265] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d: stopping tablet replica I0609 06:56:59.772912 1119 raft_consensus.cc:2147] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:59.773149 1119 raft_consensus.cc:2174] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:59.792376 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:59.792513 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:56:59.792768 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:56:59.799093 1119 master.cc:247] Master@127.1.23.254:46755 shutting down... I0609 06:56:59.811547 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 1 LEADER]: Raft consensus shutting down. I0609 06:56:59.811843 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:56:59.812175 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f: stopping tablet replica I0609 06:56:59.823261 1119 master.cc:258] Master@127.1.23.254:46755 shutdown complete. I0609 06:56:59.840647 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 96516us I0609 06:56:59.840946 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:59.841084 1119 fs_manager.cc:266] Using existing metadata directory in first data directory I0609 06:56:59.843350 1119 fs_manager.cc:399] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s I0609 06:56:59.844306 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:59.850898 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:59.851058 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/master-0-root uuid: "79487940486d4430a7133dacf260d00f" format_stamp: "Formatted at 2019-06-09 06:56:59 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:59.851328 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:56:59.902797 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:46755 I0609 06:56:59.904728 1119 webserver.cc:374] Webserver started at http://127.1.23.254:42435/ using document root and password file I0609 06:56:59.913362 3392 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f: Bootstrap starting. I0609 06:56:59.930063 3392 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates I0609 06:56:59.930650 3392 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f: Bootstrap complete. I0609 06:56:59.931937 3392 raft_consensus.cc:340] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } I0609 06:56:59.932309 3392 raft_consensus.cc:705] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79487940486d4430a7133dacf260d00f, State: Initialized, Role: FOLLOWER I0609 06:56:59.932683 3392 consensus_queue.cc:229] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } I0609 06:56:59.933034 3392 raft_consensus.cc:380] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:56:59.933190 3392 raft_consensus.cc:472] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:56:59.933413 3392 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 1 FOLLOWER]: Advancing to term 2 I0609 06:56:59.936787 3392 raft_consensus.cc:494] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } I0609 06:56:59.937146 3392 leader_election.cc:310] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79487940486d4430a7133dacf260d00f; no voters: I0609 06:56:59.938432 3394 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 2 FOLLOWER]: Leader election won for term 2 I0609 06:56:59.939028 3392 leader_election.cc:296] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [CANDIDATE]: Term 2 election: Requested vote from peers I0609 06:56:59.939679 3394 raft_consensus.cc:667] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 2 LEADER]: Becoming Leader. State: Replica: 79487940486d4430a7133dacf260d00f, State: Running, Role: LEADER I0609 06:56:59.940021 3394 consensus_queue.cc:206] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } I0609 06:56:59.941869 3392 sys_catalog.cc:338] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 2 leader_uuid: "79487940486d4430a7133dacf260d00f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } } I0609 06:56:59.942140 3395 sys_catalog.cc:338] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79487940486d4430a7133dacf260d00f. Latest consensus state: current_term: 2 leader_uuid: "79487940486d4430a7133dacf260d00f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } } I0609 06:56:59.942198 3396 sys_catalog.cc:338] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "79487940486d4430a7133dacf260d00f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79487940486d4430a7133dacf260d00f" member_type: VOTER } } I0609 06:56:59.942466 3392 sys_catalog.cc:341] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: This master's current role is: LEADER I0609 06:56:59.942749 3396 sys_catalog.cc:341] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: This master's current role is: LEADER I0609 06:56:59.942730 3395 sys_catalog.cc:341] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: This master's current role is: LEADER I0609 06:56:59.944756 3392 sys_catalog.cc:426] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [sys.catalog]: configured and running, proceeding with master startup. I0609 06:56:59.944947 3399 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:56:59.945894 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:56:59.947551 3399 catalog_manager.cc:357] Loaded metadata for table client-testtb [id=cf0e3acd9a4b4441aeee6a97086eaffe] I0609 06:56:59.949825 3399 catalog_manager.cc:411] Loaded metadata for tablet 49ea677949a8499480e011d8481e2c6a (table client-testtb [id=cf0e3acd9a4b4441aeee6a97086eaffe]) I0609 06:56:59.950457 3399 catalog_manager.cc:411] Loaded metadata for tablet 5964356d13164391b488b81a09e4f9b0 (table client-testtb [id=cf0e3acd9a4b4441aeee6a97086eaffe]) I0609 06:56:59.950909 3399 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:56:59.954725 3399 catalog_manager.cc:1076] Loading token signing keys... I0609 06:56:59.956413 3399 catalog_manager.cc:4304] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f: Loaded TSK: 0 I0609 06:56:59.975081 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 96516us I0609 06:56:59.975399 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:56:59.975543 1119 fs_manager.cc:266] Using existing metadata directory in first data directory I0609 06:56:59.978782 1119 fs_manager.cc:399] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s I0609 06:56:59.980170 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:56:59.987279 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:56:59.987428 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/ts-0-root uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" format_stamp: "Formatted at 2019-06-09 06:56:59 on dist-test-slave-dist-test-slave-9h7f" I0609 06:56:59.987704 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.ClientTest.TestBlockScannerHijackingAttempts.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:57:00.029223 1119 ts_tablet_manager.cc:344] Loading tablet metadata (0/2 complete) I0609 06:57:00.036598 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (2 total tablets, 2 live tablets) I0609 06:57:00.036733 1119 ts_tablet_manager.cc:362] Registering tablets (0/2 complete) I0609 06:57:00.040812 3411 tablet_bootstrap.cc:439] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d: Bootstrap starting. I0609 06:57:00.043628 1119 ts_tablet_manager.cc:376] Registered 2 tablets I0609 06:57:00.056851 3411 tablet_bootstrap.cc:439] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates I0609 06:57:00.057914 3411 tablet_bootstrap.cc:439] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d: Bootstrap complete. I0609 06:57:00.058310 3411 ts_tablet_manager.cc:1054] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.003s I0609 06:57:00.059389 3411 raft_consensus.cc:340] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:57:00.059757 3411 raft_consensus.cc:705] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37c1ed11cd8c4d7287c87baa7cb3db4d, State: Initialized, Role: FOLLOWER I0609 06:57:00.060199 3411 consensus_queue.cc:229] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:57:00.060591 3411 raft_consensus.cc:380] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:00.060780 3411 raft_consensus.cc:472] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:00.061007 3411 raft_consensus.cc:2890] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Advancing to term 2 I0609 06:57:00.064973 3411 raft_consensus.cc:494] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:57:00.065858 3411 leader_election.cc:310] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37c1ed11cd8c4d7287c87baa7cb3db4d; no voters: I0609 06:57:00.067399 3411 leader_election.cc:296] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [CANDIDATE]: Term 2 election: Requested vote from peers I0609 06:57:00.067697 3428 raft_consensus.cc:2644] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 2 FOLLOWER]: Leader election won for term 2 I0609 06:57:00.069643 3428 raft_consensus.cc:667] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 2 LEADER]: Becoming Leader. State: Replica: 37c1ed11cd8c4d7287c87baa7cb3db4d, State: Running, Role: LEADER I0609 06:57:00.069878 3411 ts_tablet_manager.cc:1082] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s I0609 06:57:00.070433 3411 tablet_bootstrap.cc:439] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d: Bootstrap starting. I0609 06:57:00.070286 3428 consensus_queue.cc:206] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:57:00.080770 3411 tablet_bootstrap.cc:439] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates I0609 06:57:00.081712 3411 tablet_bootstrap.cc:439] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d: Bootstrap complete. I0609 06:57:00.082304 3411 ts_tablet_manager.cc:1054] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s I0609 06:57:00.083956 3411 raft_consensus.cc:340] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:57:00.084532 3411 raft_consensus.cc:705] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37c1ed11cd8c4d7287c87baa7cb3db4d, State: Initialized, Role: FOLLOWER I0609 06:57:00.085161 3411 consensus_queue.cc:229] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:57:00.086552 3411 raft_consensus.cc:380] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:00.086843 3411 raft_consensus.cc:472] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:00.087183 3411 raft_consensus.cc:2890] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 1 FOLLOWER]: Advancing to term 2 I0609 06:57:00.094645 3411 raft_consensus.cc:494] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:57:00.095309 3411 leader_election.cc:310] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37c1ed11cd8c4d7287c87baa7cb3db4d; no voters: I0609 06:57:00.096035 3411 leader_election.cc:296] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [CANDIDATE]: Term 2 election: Requested vote from peers I0609 06:57:00.096117 3428 raft_consensus.cc:2644] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 2 FOLLOWER]: Leader election won for term 2 I0609 06:57:00.097127 3428 raft_consensus.cc:667] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 2 LEADER]: Becoming Leader. State: Replica: 37c1ed11cd8c4d7287c87baa7cb3db4d, State: Running, Role: LEADER I0609 06:57:00.097606 3411 ts_tablet_manager.cc:1082] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d: Time spent starting tablet: real 0.015s user 0.008s sys 0.004s I0609 06:57:00.097636 3428 consensus_queue.cc:206] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } } I0609 06:57:00.115589 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:45151 I0609 06:57:00.117967 1119 webserver.cc:374] Webserver started at http://127.1.23.193:42927/ using document root and password file I0609 06:57:00.129516 3474 heartbeater.cc:324] Connected to a master server at 127.1.23.254:46755 I0609 06:57:00.129735 3474 heartbeater.cc:405] Registering TS with master... I0609 06:57:00.130061 3474 heartbeater.cc:454] Master 127.1.23.254:46755 requested a full tablet report, sending... I0609 06:57:00.131925 3359 ts_manager.cc:161] Registered new tserver with Master: 37c1ed11cd8c4d7287c87baa7cb3db4d (127.1.23.193:45151) I0609 06:57:00.132674 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.012838051s I0609 06:57:00.132709 3359 catalog_manager.cc:3989] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } health_report { overall_health: HEALTHY } } } I0609 06:57:00.133860 3359 catalog_manager.cc:3989] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c1ed11cd8c4d7287c87baa7cb3db4d" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45363 } health_report { overall_health: HEALTHY } } } ================== WARNING: ThreadSanitizer: data race (pid=1119) Write of size 8 at 0x7b0c001139d0 by thread T164 (mutexes: read M470198777455412040, write M668920231272409128): #0 operator delete(void*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:119 (client-test+0x5550e1) #1 __libcpp_deallocate /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/new:236:3 (libc++.so.1+0xd7b20) #2 deallocate /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/memory:1796 (libc++.so.1+0xd7b20) #3 deallocate /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/memory:1555 (libc++.so.1+0xd7b20) #4 std::__1::basic_string, std::__1::allocator >::~basic_string() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/string:1941 (libc++.so.1+0xd7b20) #5 google::protobuf::internal::ArenaStringPtr::DestroyNoArena(std::__1::basic_string, std::__1::allocator > const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/arenastring.h:258:7 (libmaster_proto.so+0xd17a2) #6 kudu::consensus::RaftPeerPB::SharedDtor() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1039:19 (libconsensus_metadata_proto.so+0x188d3) #7 kudu::consensus::RaftPeerPB::~RaftPeerPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1035:3 (libconsensus_metadata_proto.so+0x18873) #8 kudu::consensus::RaftPeerPB::~RaftPeerPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1033:27 (libconsensus_metadata_proto.so+0x18999) #9 google::protobuf::internal::GenericTypeHandler::Delete(kudu::consensus::RaftPeerPB*, google::protobuf::Arena*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/repeated_field.h:615:7 (libconsensus.so+0xbf5d0) #10 void google::protobuf::internal::RepeatedPtrFieldBase::Destroy::TypeHandler>() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/repeated_field.h:1429:7 (libconsensus.so+0xbf559) #11 google::protobuf::RepeatedPtrField::~RepeatedPtrField() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/repeated_field.h:1892:3 (libconsensus.so+0xbe75a) #12 kudu::consensus::RaftConfigPB::~RaftConfigPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1699:1 (libconsensus_metadata_proto.so+0x19d14) #13 kudu::consensus::RaftConfigPB::~RaftConfigPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1696:31 (libconsensus_metadata_proto.so+0x19d69) #14 kudu::consensus::ConsensusStatePB::SharedDtor() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:2189:44 (libconsensus_metadata_proto.so+0x1ae20) #15 kudu::consensus::ConsensusStatePB::~ConsensusStatePB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:2184:3 (libconsensus_metadata_proto.so+0x1ad93) #16 kudu::consensus::ConsensusStatePB::~ConsensusStatePB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:2182:39 (libconsensus_metadata_proto.so+0x1ae89) #17 kudu::master::SysTabletsEntryPB::SharedDtor() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.pb.cc:3181:44 (libmaster_proto.so+0x922b3) #18 kudu::master::SysTabletsEntryPB::~SysTabletsEntryPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.pb.cc:3173:3 (libmaster_proto.so+0x921f3) #19 kudu::master::PersistentTabletInfo::~PersistentTabletInfo() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.h:114:8 (libmaster.so+0x13f919) #20 std::__1::default_delete::operator()(kudu::master::PersistentTabletInfo*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/memory:2285:5 (libmaster.so+0x116ea3) #21 std::__1::unique_ptr >::reset(kudu::master::PersistentTabletInfo*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/memory:2598 (libmaster.so+0x116ea3) #22 kudu::CowObject::CommitMutation() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/cow_object.h:93 (libmaster.so+0x116ea3) #23 kudu::CowGroupLock, std::__1::allocator >, kudu::master::PersistentTabletInfo>::Commit() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/cow_object.h:386:17 (libmaster.so+0x119031) #24 kudu::master::CatalogManager::ProcessTabletReport(kudu::master::TSDescriptor*, kudu::master::TabletReportPB const&, kudu::master::TabletReportUpdatesPB*, kudu::rpc::RpcContext*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.cc:4124:16 (libmaster.so+0x106ece) #25 kudu::master::MasterServiceImpl::TSHeartbeat(kudu::master::TSHeartbeatRequestPB const*, kudu::master::TSHeartbeatResponsePB*, kudu::rpc::RpcContext*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master_service.cc:257:44 (libmaster.so+0x1776c5) #26 kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_1::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.service.cc:145:13 (libmaster_proto.so+0xf21b4) #27 _ZNSt3__18__invokeIRZN4kudu6master15MasterServiceIfC1ERK13scoped_refptrINS1_12MetricEntityEERKS4_INS1_3rpc13ResultTrackerEEE3$_1JPKN6google8protobuf7MessageEPSI_PNS9_10RpcContextEEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSO_DpOSP_ /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:4482:1 (libmaster_proto.so+0xf2141) #28 void std::__1::__invoke_void_return_wrapper::__call const&, scoped_refptr const&)::$_1&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*>(kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_1&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/__functional_base:349 (libmaster_proto.so+0xf2141) #29 std::__1::__function::__func const&, scoped_refptr const&)::$_1, std::__1::allocator const&, scoped_refptr const&)::$_1>, void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1562:12 (libmaster_proto.so+0xf2054) #30 std::__1::function::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1916:12 (libkrpc.so+0xf2069) #31 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:139:3 (libkrpc.so+0xf1a05) #32 kudu::rpc::ServicePool::RunThread() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:225:15 (libkrpc.so+0xf2f0c) #33 boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/mem_fn_template.hpp:49:29 (libkrpc.so+0xf4e19) #34 void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:259:9 (libkrpc.so+0xf4d6a) #35 boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:1222:16 (libkrpc.so+0xf4cf3) #36 boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:159:11 (libkrpc.so+0xf4ae9) #37 boost::function0::operator()() const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:770:14 (libkrpc.so+0xb8411) #38 kudu::Thread::SuperviseThread(void*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:657:3 (libkudu_util.so+0x1e17d4) Previous read of size 8 at 0x7b0c001139d0 by thread T58 (mutexes: read M470198777455412040): #0 strings::memeq(void const*, void const*, unsigned long) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/strings/fastmem.h:46:36 (client-test+0x5ee81a) #1 operator==(StringPiece, StringPiece) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/strings/stringpiece.h:314:7 (client-test+0x5fb371) #2 std::__1::equal_to::operator()(StringPiece const&, StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:670:21 (libmaster.so+0x1387d7) #3 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::KeyInfo::equals(StringPiece const&, StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:1287 (libmaster.so+0x1387d7) #4 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::equals(StringPiece const&, StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:1299:21 (libmaster.so+0x13864d) #5 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::find_position(StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:879:18 (libmaster.so+0x14ab8b) #6 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::find(StringPiece const&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:892:43 (libmaster.so+0x14a964) #7 google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::find(StringPiece const&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/dense_hash_map:249:51 (libmaster.so+0x14a7f9) #8 std::__1::pair, std::__1::equal_to, google::libc_allocator_with_realloc > >::mapped_type* const, bool> ComputeIfAbsentReturnAbsense, std::__1::equal_to, google::libc_allocator_with_realloc > >, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16>(google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >*, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::key_type const&, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/map-util.h:883:53 (libmaster.so+0x11080a) #9 google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::mapped_type* const ComputeIfAbsent, std::__1::equal_to, google::libc_allocator_with_realloc > >, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16>(google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >*, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::key_type const&, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/map-util.h:906:10 (libmaster.so+0x10abb0) #10 kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.cc:4680:18 (libmaster.so+0x10a7aa) #11 kudu::master::CatalogManager::GetTableLocations(kudu::master::GetTableLocationsRequestPB const*, kudu::master::GetTableLocationsResponsePB*, boost::optional, std::__1::allocator > const&>) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.cc:4855:16 (libmaster.so+0x10bf48) #12 kudu::master::MasterServiceImpl::GetTableLocations(kudu::master::GetTableLocationsRequestPB const*, kudu::master::GetTableLocationsResponsePB*, kudu::rpc::RpcContext*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master_service.cc:424:42 (libmaster.so+0x178b65) #13 kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_19::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.service.cc:325:13 (libmaster_proto.so+0xf61d7) #14 _ZNSt3__18__invokeIRZN4kudu6master15MasterServiceIfC1ERK13scoped_refptrINS1_12MetricEntityEERKS4_INS1_3rpc13ResultTrackerEEE4$_19JPKN6google8protobuf7MessageEPSI_PNS9_10RpcContextEEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSO_DpOSP_ /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:4482:1 (libmaster_proto.so+0xf6161) #15 void std::__1::__invoke_void_return_wrapper::__call const&, scoped_refptr const&)::$_19&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*>(kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_19&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/__functional_base:349 (libmaster_proto.so+0xf6161) #16 std::__1::__function::__func const&, scoped_refptr const&)::$_19, std::__1::allocator const&, scoped_refptr const&)::$_19>, void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1562:12 (libmaster_proto.so+0xf6074) #17 std::__1::function::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1916:12 (libkrpc.so+0xf2069) #18 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:139:3 (libkrpc.so+0xf1a05) #19 kudu::rpc::ServicePool::RunThread() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:225:15 (libkrpc.so+0xf2f0c) #20 boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/mem_fn_template.hpp:49:29 (libkrpc.so+0xf4e19) #21 void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:259:9 (libkrpc.so+0xf4d6a) #22 boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:1222:16 (libkrpc.so+0xf4cf3) #23 boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:159:11 (libkrpc.so+0xf4ae9) #24 boost::function0::operator()() const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:770:14 (libkrpc.so+0xb8411) #25 kudu::Thread::SuperviseThread(void*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:657:3 (libkudu_util.so+0x1e17d4) Mutex M470198777455412040 is already destroyed. Mutex M668920231272409128 is already destroyed. Thread T164 'rpc worker-3359' (tid=3359, running) created by main thread at: #0 pthread_create /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:992 (client-test+0x4e762b) #1 kudu::Thread::StartThread(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, boost::function const&, unsigned long, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:601:15 (libkudu_util.so+0x1e0fbb) #2 kudu::Status kudu::Thread::Create(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, void (kudu::rpc::ServicePool::* const&)(), kudu::rpc::ServicePool* const&, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:164:12 (libkrpc.so+0xf4205) #3 kudu::rpc::ServicePool::Init(int) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5 (libkrpc.so+0xf2b16) #4 kudu::RpcServer::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/rpc_server.cc:157:3 (libserver_process.so+0x7a799) #5 kudu::server::ServerBase::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/server_base.cc:620:23 (libserver_process.so+0x81aef) #6 kudu::master::Master::StartAsync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master.cc:186:3 (libmaster.so+0x160ddf) #7 kudu::master::MiniMaster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/mini_master.cc:94:3 (libmaster.so+0x17e254) #8 kudu::cluster::InternalMiniCluster::StartMasters() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:160:5 (libmini_cluster.so+0x409d3) #9 kudu::cluster::InternalMiniCluster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:99:3 (libmini_cluster.so+0x404ad) #10 kudu::cluster::InternalMiniCluster::StartSync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:180:3 (libmini_cluster.so+0x41984) #11 kudu::client::ClientTest_TestBlockScannerHijackingAttempts_Test::TestBody() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/client-test.cc:5884:3 (client-test+0x5c89db) #12 void testing::internal::HandleSehExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x5524f) #13 void testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x5524f) #14 testing::Test::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2474:5 (libgmock.so+0x345f8) #15 testing::TestInfo::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2656:11 (libgmock.so+0x3588c) #16 testing::TestCase::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2774:28 (libgmock.so+0x36366) #17 testing::internal::UnitTestImpl::RunAllTests() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4649:43 (libgmock.so+0x4265a) #18 bool testing::internal::HandleSehExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x561bf) #19 bool testing::internal::HandleExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x561bf) #20 testing::UnitTest::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4257:10 (libgmock.so+0x41f42) #21 RUN_ALL_TESTS() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2233:46 (libkudu_test_main.so+0x34db) #22 main /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:106:13 (libkudu_test_main.so+0x2b66) Thread T58 'rpc worker-3353' (tid=3353, running) created by main thread at: #0 pthread_create /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:992 (client-test+0x4e762b) #1 kudu::Thread::StartThread(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, boost::function const&, unsigned long, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:601:15 (libkudu_util.so+0x1e0fbb) #2 kudu::Status kudu::Thread::Create(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, void (kudu::rpc::ServicePool::* const&)(), kudu::rpc::ServicePool* const&, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:164:12 (libkrpc.so+0xf4205) #3 kudu::rpc::ServicePool::Init(int) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5 (libkrpc.so+0xf2b16) #4 kudu::RpcServer::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/rpc_server.cc:157:3 (libserver_process.so+0x7a799) #5 kudu::server::ServerBase::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/server_base.cc:620:23 (libserver_process.so+0x81aef) #6 kudu::master::Master::StartAsync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master.cc:186:3 (libmaster.so+0x160ddf) #7 kudu::master::MiniMaster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/mini_master.cc:94:3 (libmaster.so+0x17e254) #8 kudu::cluster::InternalMiniCluster::StartMasters() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:160:5 (libmini_cluster.so+0x409d3) #9 kudu::cluster::InternalMiniCluster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:99:3 (libmini_cluster.so+0x404ad) #10 kudu::cluster::InternalMiniCluster::StartSync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:180:3 (libmini_cluster.so+0x41984) #11 kudu::client::ClientTest_TestBlockScannerHijackingAttempts_Test::TestBody() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/client-test.cc:5884:3 (client-test+0x5c89db) #12 void testing::internal::HandleSehExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x5524f) #13 void testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x5524f) #14 testing::Test::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2474:5 (libgmock.so+0x345f8) #15 testing::TestInfo::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2656:11 (libgmock.so+0x3588c) #16 testing::TestCase::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2774:28 (libgmock.so+0x36366) #17 testing::internal::UnitTestImpl::RunAllTests() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4649:43 (libgmock.so+0x4265a) #18 bool testing::internal::HandleSehExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x561bf) #19 bool testing::internal::HandleExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x561bf) #20 testing::UnitTest::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4257:10 (libgmock.so+0x41f42) #21 RUN_ALL_TESTS() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2233:46 (libkudu_test_main.so+0x34db) #22 main /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:106:13 (libkudu_test_main.so+0x2b66) SUMMARY: ThreadSanitizer: data race /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:119 in operator delete(void*) ================== ================== WARNING: ThreadSanitizer: data race (pid=1119) Write of size 8 at 0x7b0c001139d8 by thread T164 (mutexes: read M470198777455412040, write M668920231272409128): #0 operator delete(void*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:119 (client-test+0x5550e1) #1 __libcpp_deallocate /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/new:236:3 (libc++.so.1+0xd7b20) #2 deallocate /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/memory:1796 (libc++.so.1+0xd7b20) #3 deallocate /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/memory:1555 (libc++.so.1+0xd7b20) #4 std::__1::basic_string, std::__1::allocator >::~basic_string() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/string:1941 (libc++.so.1+0xd7b20) #5 google::protobuf::internal::ArenaStringPtr::DestroyNoArena(std::__1::basic_string, std::__1::allocator > const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/arenastring.h:258:7 (libmaster_proto.so+0xd17a2) #6 kudu::consensus::RaftPeerPB::SharedDtor() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1039:19 (libconsensus_metadata_proto.so+0x188d3) #7 kudu::consensus::RaftPeerPB::~RaftPeerPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1035:3 (libconsensus_metadata_proto.so+0x18873) #8 kudu::consensus::RaftPeerPB::~RaftPeerPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1033:27 (libconsensus_metadata_proto.so+0x18999) #9 google::protobuf::internal::GenericTypeHandler::Delete(kudu::consensus::RaftPeerPB*, google::protobuf::Arena*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/repeated_field.h:615:7 (libconsensus.so+0xbf5d0) #10 void google::protobuf::internal::RepeatedPtrFieldBase::Destroy::TypeHandler>() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/repeated_field.h:1429:7 (libconsensus.so+0xbf559) #11 google::protobuf::RepeatedPtrField::~RepeatedPtrField() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/repeated_field.h:1892:3 (libconsensus.so+0xbe75a) #12 kudu::consensus::RaftConfigPB::~RaftConfigPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1699:1 (libconsensus_metadata_proto.so+0x19d14) #13 kudu::consensus::RaftConfigPB::~RaftConfigPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1696:31 (libconsensus_metadata_proto.so+0x19d69) #14 kudu::consensus::ConsensusStatePB::SharedDtor() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:2189:44 (libconsensus_metadata_proto.so+0x1ae20) #15 kudu::consensus::ConsensusStatePB::~ConsensusStatePB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:2184:3 (libconsensus_metadata_proto.so+0x1ad93) #16 kudu::consensus::ConsensusStatePB::~ConsensusStatePB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:2182:39 (libconsensus_metadata_proto.so+0x1ae89) #17 kudu::master::SysTabletsEntryPB::SharedDtor() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.pb.cc:3181:44 (libmaster_proto.so+0x922b3) #18 kudu::master::SysTabletsEntryPB::~SysTabletsEntryPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.pb.cc:3173:3 (libmaster_proto.so+0x921f3) #19 kudu::master::PersistentTabletInfo::~PersistentTabletInfo() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.h:114:8 (libmaster.so+0x13f919) #20 std::__1::default_delete::operator()(kudu::master::PersistentTabletInfo*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/memory:2285:5 (libmaster.so+0x116ea3) #21 std::__1::unique_ptr >::reset(kudu::master::PersistentTabletInfo*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/memory:2598 (libmaster.so+0x116ea3) #22 kudu::CowObject::CommitMutation() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/cow_object.h:93 (libmaster.so+0x116ea3) #23 kudu::CowGroupLock, std::__1::allocator >, kudu::master::PersistentTabletInfo>::Commit() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/cow_object.h:386:17 (libmaster.so+0x119031) #24 kudu::master::CatalogManager::ProcessTabletReport(kudu::master::TSDescriptor*, kudu::master::TabletReportPB const&, kudu::master::TabletReportUpdatesPB*, kudu::rpc::RpcContext*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.cc:4124:16 (libmaster.so+0x106ece) #25 kudu::master::MasterServiceImpl::TSHeartbeat(kudu::master::TSHeartbeatRequestPB const*, kudu::master::TSHeartbeatResponsePB*, kudu::rpc::RpcContext*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master_service.cc:257:44 (libmaster.so+0x1776c5) #26 kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_1::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.service.cc:145:13 (libmaster_proto.so+0xf21b4) #27 _ZNSt3__18__invokeIRZN4kudu6master15MasterServiceIfC1ERK13scoped_refptrINS1_12MetricEntityEERKS4_INS1_3rpc13ResultTrackerEEE3$_1JPKN6google8protobuf7MessageEPSI_PNS9_10RpcContextEEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSO_DpOSP_ /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:4482:1 (libmaster_proto.so+0xf2141) #28 void std::__1::__invoke_void_return_wrapper::__call const&, scoped_refptr const&)::$_1&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*>(kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_1&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/__functional_base:349 (libmaster_proto.so+0xf2141) #29 std::__1::__function::__func const&, scoped_refptr const&)::$_1, std::__1::allocator const&, scoped_refptr const&)::$_1>, void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1562:12 (libmaster_proto.so+0xf2054) #30 std::__1::function::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1916:12 (libkrpc.so+0xf2069) #31 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:139:3 (libkrpc.so+0xf1a05) #32 kudu::rpc::ServicePool::RunThread() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:225:15 (libkrpc.so+0xf2f0c) #33 boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/mem_fn_template.hpp:49:29 (libkrpc.so+0xf4e19) #34 void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:259:9 (libkrpc.so+0xf4d6a) #35 boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:1222:16 (libkrpc.so+0xf4cf3) #36 boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:159:11 (libkrpc.so+0xf4ae9) #37 boost::function0::operator()() const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:770:14 (libkrpc.so+0xb8411) #38 kudu::Thread::SuperviseThread(void*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:657:3 (libkudu_util.so+0x1e17d4) Previous read of size 8 at 0x7b0c001139d8 by thread T58 (mutexes: read M470198777455412040): #0 strings::memeq(void const*, void const*, unsigned long) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/strings/fastmem.h:61:38 (client-test+0x5ee8b1) #1 operator==(StringPiece, StringPiece) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/strings/stringpiece.h:314:7 (client-test+0x5fb371) #2 std::__1::equal_to::operator()(StringPiece const&, StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:670:21 (libmaster.so+0x1387d7) #3 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::KeyInfo::equals(StringPiece const&, StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:1287 (libmaster.so+0x1387d7) #4 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::equals(StringPiece const&, StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:1299:21 (libmaster.so+0x13864d) #5 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::find_position(StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:879:18 (libmaster.so+0x14ab8b) #6 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::find(StringPiece const&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:892:43 (libmaster.so+0x14a964) #7 google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::find(StringPiece const&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/dense_hash_map:249:51 (libmaster.so+0x14a7f9) #8 std::__1::pair, std::__1::equal_to, google::libc_allocator_with_realloc > >::mapped_type* const, bool> ComputeIfAbsentReturnAbsense, std::__1::equal_to, google::libc_allocator_with_realloc > >, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16>(google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >*, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::key_type const&, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/map-util.h:883:53 (libmaster.so+0x11080a) #9 google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::mapped_type* const ComputeIfAbsent, std::__1::equal_to, google::libc_allocator_with_realloc > >, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16>(google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >*, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::key_type const&, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/map-util.h:906:10 (libmaster.so+0x10abb0) #10 kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.cc:4680:18 (libmaster.so+0x10a7aa) #11 kudu::master::CatalogManager::GetTableLocations(kudu::master::GetTableLocationsRequestPB const*, kudu::master::GetTableLocationsResponsePB*, boost::optional, std::__1::allocator > const&>) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.cc:4855:16 (libmaster.so+0x10bf48) #12 kudu::master::MasterServiceImpl::GetTableLocations(kudu::master::GetTableLocationsRequestPB const*, kudu::master::GetTableLocationsResponsePB*, kudu::rpc::RpcContext*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master_service.cc:424:42 (libmaster.so+0x178b65) #13 kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_19::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.service.cc:325:13 (libmaster_proto.so+0xf61d7) #14 _ZNSt3__18__invokeIRZN4kudu6master15MasterServiceIfC1ERK13scoped_refptrINS1_12MetricEntityEERKS4_INS1_3rpc13ResultTrackerEEE4$_19JPKN6google8protobuf7MessageEPSI_PNS9_10RpcContextEEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSO_DpOSP_ /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:4482:1 (libmaster_proto.so+0xf6161) #15 void std::__1::__invoke_void_return_wrapper::__call const&, scoped_refptr const&)::$_19&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*>(kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_19&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/__functional_base:349 (libmaster_proto.so+0xf6161) #16 std::__1::__function::__func const&, scoped_refptr const&)::$_19, std::__1::allocator const&, scoped_refptr const&)::$_19>, void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1562:12 (libmaster_proto.so+0xf6074) #17 std::__1::function::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1916:12 (libkrpc.so+0xf2069) #18 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:139:3 (libkrpc.so+0xf1a05) #19 kudu::rpc::ServicePool::RunThread() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:225:15 (libkrpc.so+0xf2f0c) #20 boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/mem_fn_template.hpp:49:29 (libkrpc.so+0xf4e19) #21 void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:259:9 (libkrpc.so+0xf4d6a) #22 boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:1222:16 (libkrpc.so+0xf4cf3) #23 boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:159:11 (libkrpc.so+0xf4ae9) #24 boost::function0::operator()() const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:770:14 (libkrpc.so+0xb8411) #25 kudu::Thread::SuperviseThread(void*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:657:3 (libkudu_util.so+0x1e17d4) Mutex M470198777455412040 is already destroyed. Mutex M668920231272409128 is already destroyed. Thread T164 'rpc worker-3359' (tid=3359, running) created by main thread at: #0 pthread_create /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:992 (client-test+0x4e762b) #1 kudu::Thread::StartThread(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, boost::function const&, unsigned long, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:601:15 (libkudu_util.so+0x1e0fbb) #2 kudu::Status kudu::Thread::Create(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, void (kudu::rpc::ServicePool::* const&)(), kudu::rpc::ServicePool* const&, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:164:12 (libkrpc.so+0xf4205) #3 kudu::rpc::ServicePool::Init(int) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5 (libkrpc.so+0xf2b16) #4 kudu::RpcServer::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/rpc_server.cc:157:3 (libserver_process.so+0x7a799) #5 kudu::server::ServerBase::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/server_base.cc:620:23 (libserver_process.so+0x81aef) #6 kudu::master::Master::StartAsync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master.cc:186:3 (libmaster.so+0x160ddf) #7 kudu::master::MiniMaster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/mini_master.cc:94:3 (libmaster.so+0x17e254) #8 kudu::cluster::InternalMiniCluster::StartMasters() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:160:5 (libmini_cluster.so+0x409d3) #9 kudu::cluster::InternalMiniCluster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:99:3 (libmini_cluster.so+0x404ad) #10 kudu::cluster::InternalMiniCluster::StartSync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:180:3 (libmini_cluster.so+0x41984) #11 kudu::client::ClientTest_TestBlockScannerHijackingAttempts_Test::TestBody() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/client-test.cc:5884:3 (client-test+0x5c89db) #12 void testing::internal::HandleSehExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x5524f) #13 void testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x5524f) #14 testing::Test::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2474:5 (libgmock.so+0x345f8) #15 testing::TestInfo::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2656:11 (libgmock.so+0x3588c) #16 testing::TestCase::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2774:28 (libgmock.so+0x36366) #17 testing::internal::UnitTestImpl::RunAllTests() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4649:43 (libgmock.so+0x4265a) #18 bool testing::internal::HandleSehExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x561bf) #19 bool testing::internal::HandleExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x561bf) #20 testing::UnitTest::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4257:10 (libgmock.so+0x41f42) #21 RUN_ALL_TESTS() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2233:46 (libkudu_test_main.so+0x34db) #22 main /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:106:13 (libkudu_test_main.so+0x2b66) Thread T58 'rpc worker-3353' (tid=3353, running) created by main thread at: #0 pthread_create /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:992 (client-test+0x4e762b) #1 kudu::Thread::StartThread(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, boost::function const&, unsigned long, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:601:15 (libkudu_util.so+0x1e0fbb) #2 kudu::Status kudu::Thread::Create(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, void (kudu::rpc::ServicePool::* const&)(), kudu::rpc::ServicePool* const&, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:164:12 (libkrpc.so+0xf4205) #3 kudu::rpc::ServicePool::Init(int) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5 (libkrpc.so+0xf2b16) #4 kudu::RpcServer::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/rpc_server.cc:157:3 (libserver_process.so+0x7a799) #5 kudu::server::ServerBase::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/server_base.cc:620:23 (libserver_process.so+0x81aef) #6 kudu::master::Master::StartAsync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master.cc:186:3 (libmaster.so+0x160ddf) #7 kudu::master::MiniMaster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/mini_master.cc:94:3 (libmaster.so+0x17e254) #8 kudu::cluster::InternalMiniCluster::StartMasters() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:160:5 (libmini_cluster.so+0x409d3) #9 kudu::cluster::InternalMiniCluster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:99:3 (libmini_cluster.so+0x404ad) #10 kudu::cluster::InternalMiniCluster::StartSync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:180:3 (libmini_cluster.so+0x41984) #11 kudu::client::ClientTest_TestBlockScannerHijackingAttempts_Test::TestBody() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/client-test.cc:5884:3 (client-test+0x5c89db) #12 void testing::internal::HandleSehExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x5524f) #13 void testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x5524f) #14 testing::Test::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2474:5 (libgmock.so+0x345f8) #15 testing::TestInfo::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2656:11 (libgmock.so+0x3588c) #16 testing::TestCase::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2774:28 (libgmock.so+0x36366) #17 testing::internal::UnitTestImpl::RunAllTests() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4649:43 (libgmock.so+0x4265a) #18 bool testing::internal::HandleSehExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x561bf) #19 bool testing::internal::HandleExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x561bf) #20 testing::UnitTest::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4257:10 (libgmock.so+0x41f42) #21 RUN_ALL_TESTS() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2233:46 (libkudu_test_main.so+0x34db) #22 main /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:106:13 (libkudu_test_main.so+0x2b66) SUMMARY: ThreadSanitizer: data race /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:119 in operator delete(void*) ================== ================== WARNING: ThreadSanitizer: data race (pid=1119) Write of size 8 at 0x7b0c001139e0 by thread T164 (mutexes: read M470198777455412040, write M668920231272409128): #0 operator delete(void*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:119 (client-test+0x5550e1) #1 __libcpp_deallocate /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/new:236:3 (libc++.so.1+0xd7b20) #2 deallocate /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/memory:1796 (libc++.so.1+0xd7b20) #3 deallocate /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/memory:1555 (libc++.so.1+0xd7b20) #4 std::__1::basic_string, std::__1::allocator >::~basic_string() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/string:1941 (libc++.so.1+0xd7b20) #5 google::protobuf::internal::ArenaStringPtr::DestroyNoArena(std::__1::basic_string, std::__1::allocator > const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/arenastring.h:258:7 (libmaster_proto.so+0xd17a2) #6 kudu::consensus::RaftPeerPB::SharedDtor() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1039:19 (libconsensus_metadata_proto.so+0x188d3) #7 kudu::consensus::RaftPeerPB::~RaftPeerPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1035:3 (libconsensus_metadata_proto.so+0x18873) #8 kudu::consensus::RaftPeerPB::~RaftPeerPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1033:27 (libconsensus_metadata_proto.so+0x18999) #9 google::protobuf::internal::GenericTypeHandler::Delete(kudu::consensus::RaftPeerPB*, google::protobuf::Arena*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/repeated_field.h:615:7 (libconsensus.so+0xbf5d0) #10 void google::protobuf::internal::RepeatedPtrFieldBase::Destroy::TypeHandler>() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/repeated_field.h:1429:7 (libconsensus.so+0xbf559) #11 google::protobuf::RepeatedPtrField::~RepeatedPtrField() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/repeated_field.h:1892:3 (libconsensus.so+0xbe75a) #12 kudu::consensus::RaftConfigPB::~RaftConfigPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1699:1 (libconsensus_metadata_proto.so+0x19d14) #13 kudu::consensus::RaftConfigPB::~RaftConfigPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1696:31 (libconsensus_metadata_proto.so+0x19d69) #14 kudu::consensus::ConsensusStatePB::SharedDtor() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:2189:44 (libconsensus_metadata_proto.so+0x1ae20) #15 kudu::consensus::ConsensusStatePB::~ConsensusStatePB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:2184:3 (libconsensus_metadata_proto.so+0x1ad93) #16 kudu::consensus::ConsensusStatePB::~ConsensusStatePB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:2182:39 (libconsensus_metadata_proto.so+0x1ae89) #17 kudu::master::SysTabletsEntryPB::SharedDtor() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.pb.cc:3181:44 (libmaster_proto.so+0x922b3) #18 kudu::master::SysTabletsEntryPB::~SysTabletsEntryPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.pb.cc:3173:3 (libmaster_proto.so+0x921f3) #19 kudu::master::PersistentTabletInfo::~PersistentTabletInfo() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.h:114:8 (libmaster.so+0x13f919) #20 std::__1::default_delete::operator()(kudu::master::PersistentTabletInfo*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/memory:2285:5 (libmaster.so+0x116ea3) #21 std::__1::unique_ptr >::reset(kudu::master::PersistentTabletInfo*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/memory:2598 (libmaster.so+0x116ea3) #22 kudu::CowObject::CommitMutation() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/cow_object.h:93 (libmaster.so+0x116ea3) #23 kudu::CowGroupLock, std::__1::allocator >, kudu::master::PersistentTabletInfo>::Commit() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/cow_object.h:386:17 (libmaster.so+0x119031) #24 kudu::master::CatalogManager::ProcessTabletReport(kudu::master::TSDescriptor*, kudu::master::TabletReportPB const&, kudu::master::TabletReportUpdatesPB*, kudu::rpc::RpcContext*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.cc:4124:16 (libmaster.so+0x106ece) #25 kudu::master::MasterServiceImpl::TSHeartbeat(kudu::master::TSHeartbeatRequestPB const*, kudu::master::TSHeartbeatResponsePB*, kudu::rpc::RpcContext*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master_service.cc:257:44 (libmaster.so+0x1776c5) #26 kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_1::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.service.cc:145:13 (libmaster_proto.so+0xf21b4) #27 _ZNSt3__18__invokeIRZN4kudu6master15MasterServiceIfC1ERK13scoped_refptrINS1_12MetricEntityEERKS4_INS1_3rpc13ResultTrackerEEE3$_1JPKN6google8protobuf7MessageEPSI_PNS9_10RpcContextEEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSO_DpOSP_ /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:4482:1 (libmaster_proto.so+0xf2141) #28 void std::__1::__invoke_void_return_wrapper::__call const&, scoped_refptr const&)::$_1&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*>(kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_1&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/__functional_base:349 (libmaster_proto.so+0xf2141) #29 std::__1::__function::__func const&, scoped_refptr const&)::$_1, std::__1::allocator const&, scoped_refptr const&)::$_1>, void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1562:12 (libmaster_proto.so+0xf2054) #30 std::__1::function::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1916:12 (libkrpc.so+0xf2069) #31 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:139:3 (libkrpc.so+0xf1a05) #32 kudu::rpc::ServicePool::RunThread() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:225:15 (libkrpc.so+0xf2f0c) #33 boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/mem_fn_template.hpp:49:29 (libkrpc.so+0xf4e19) #34 void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:259:9 (libkrpc.so+0xf4d6a) #35 boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:1222:16 (libkrpc.so+0xf4cf3) #36 boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:159:11 (libkrpc.so+0xf4ae9) #37 boost::function0::operator()() const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:770:14 (libkrpc.so+0xb8411) #38 kudu::Thread::SuperviseThread(void*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:657:3 (libkudu_util.so+0x1e17d4) Previous read of size 8 at 0x7b0c001139e0 by thread T58 (mutexes: read M470198777455412040): #0 strings::memeq(void const*, void const*, unsigned long) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/strings/fastmem.h:62:42 (client-test+0x5ee8d7) #1 operator==(StringPiece, StringPiece) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/strings/stringpiece.h:314:7 (client-test+0x5fb371) #2 std::__1::equal_to::operator()(StringPiece const&, StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:670:21 (libmaster.so+0x1387d7) #3 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::KeyInfo::equals(StringPiece const&, StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:1287 (libmaster.so+0x1387d7) #4 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::equals(StringPiece const&, StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:1299:21 (libmaster.so+0x13864d) #5 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::find_position(StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:879:18 (libmaster.so+0x14ab8b) #6 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::find(StringPiece const&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:892:43 (libmaster.so+0x14a964) #7 google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::find(StringPiece const&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/dense_hash_map:249:51 (libmaster.so+0x14a7f9) #8 std::__1::pair, std::__1::equal_to, google::libc_allocator_with_realloc > >::mapped_type* const, bool> ComputeIfAbsentReturnAbsense, std::__1::equal_to, google::libc_allocator_with_realloc > >, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16>(google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >*, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::key_type const&, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/map-util.h:883:53 (libmaster.so+0x11080a) #9 google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::mapped_type* const ComputeIfAbsent, std::__1::equal_to, google::libc_allocator_with_realloc > >, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16>(google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >*, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::key_type const&, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/map-util.h:906:10 (libmaster.so+0x10abb0) #10 kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.cc:4680:18 (libmaster.so+0x10a7aa) #11 kudu::master::CatalogManager::GetTableLocations(kudu::master::GetTableLocationsRequestPB const*, kudu::master::GetTableLocationsResponsePB*, boost::optional, std::__1::allocator > const&>) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.cc:4855:16 (libmaster.so+0x10bf48) #12 kudu::master::MasterServiceImpl::GetTableLocations(kudu::master::GetTableLocationsRequestPB const*, kudu::master::GetTableLocationsResponsePB*, kudu::rpc::RpcContext*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master_service.cc:424:42 (libmaster.so+0x178b65) #13 kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_19::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.service.cc:325:13 (libmaster_proto.so+0xf61d7) #14 _ZNSt3__18__invokeIRZN4kudu6master15MasterServiceIfC1ERK13scoped_refptrINS1_12MetricEntityEERKS4_INS1_3rpc13ResultTrackerEEE4$_19JPKN6google8protobuf7MessageEPSI_PNS9_10RpcContextEEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSO_DpOSP_ /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:4482:1 (libmaster_proto.so+0xf6161) #15 void std::__1::__invoke_void_return_wrapper::__call const&, scoped_refptr const&)::$_19&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*>(kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_19&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/__functional_base:349 (libmaster_proto.so+0xf6161) #16 std::__1::__function::__func const&, scoped_refptr const&)::$_19, std::__1::allocator const&, scoped_refptr const&)::$_19>, void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1562:12 (libmaster_proto.so+0xf6074) #17 std::__1::function::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1916:12 (libkrpc.so+0xf2069) #18 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:139:3 (libkrpc.so+0xf1a05) #19 kudu::rpc::ServicePool::RunThread() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:225:15 (libkrpc.so+0xf2f0c) #20 boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/mem_fn_template.hpp:49:29 (libkrpc.so+0xf4e19) #21 void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:259:9 (libkrpc.so+0xf4d6a) #22 boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:1222:16 (libkrpc.so+0xf4cf3) #23 boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:159:11 (libkrpc.so+0xf4ae9) #24 boost::function0::operator()() const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:770:14 (libkrpc.so+0xb8411) #25 kudu::Thread::SuperviseThread(void*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:657:3 (libkudu_util.so+0x1e17d4) Mutex M470198777455412040 is already destroyed. Mutex M668920231272409128 is already destroyed. Thread T164 'rpc worker-3359' (tid=3359, running) created by main thread at: #0 pthread_create /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:992 (client-test+0x4e762b) #1 kudu::Thread::StartThread(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, boost::function const&, unsigned long, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:601:15 (libkudu_util.so+0x1e0fbb) #2 kudu::Status kudu::Thread::Create(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, void (kudu::rpc::ServicePool::* const&)(), kudu::rpc::ServicePool* const&, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:164:12 (libkrpc.so+0xf4205) #3 kudu::rpc::ServicePool::Init(int) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5 (libkrpc.so+0xf2b16) #4 kudu::RpcServer::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/rpc_server.cc:157:3 (libserver_process.so+0x7a799) #5 kudu::server::ServerBase::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/server_base.cc:620:23 (libserver_process.so+0x81aef) #6 kudu::master::Master::StartAsync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master.cc:186:3 (libmaster.so+0x160ddf) #7 kudu::master::MiniMaster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/mini_master.cc:94:3 (libmaster.so+0x17e254) #8 kudu::cluster::InternalMiniCluster::StartMasters() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:160:5 (libmini_cluster.so+0x409d3) #9 kudu::cluster::InternalMiniCluster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:99:3 (libmini_cluster.so+0x404ad) #10 kudu::cluster::InternalMiniCluster::StartSync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:180:3 (libmini_cluster.so+0x41984) #11 kudu::client::ClientTest_TestBlockScannerHijackingAttempts_Test::TestBody() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/client-test.cc:5884:3 (client-test+0x5c89db) #12 void testing::internal::HandleSehExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x5524f) #13 void testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x5524f) #14 testing::Test::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2474:5 (libgmock.so+0x345f8) #15 testing::TestInfo::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2656:11 (libgmock.so+0x3588c) #16 testing::TestCase::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2774:28 (libgmock.so+0x36366) #17 testing::internal::UnitTestImpl::RunAllTests() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4649:43 (libgmock.so+0x4265a) #18 bool testing::internal::HandleSehExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x561bf) #19 bool testing::internal::HandleExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x561bf) #20 testing::UnitTest::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4257:10 (libgmock.so+0x41f42) #21 RUN_ALL_TESTS() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2233:46 (libkudu_test_main.so+0x34db) #22 main /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:106:13 (libkudu_test_main.so+0x2b66) Thread T58 'rpc worker-3353' (tid=3353, running) created by main thread at: #0 pthread_create /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:992 (client-test+0x4e762b) #1 kudu::Thread::StartThread(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, boost::function const&, unsigned long, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:601:15 (libkudu_util.so+0x1e0fbb) #2 kudu::Status kudu::Thread::Create(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, void (kudu::rpc::ServicePool::* const&)(), kudu::rpc::ServicePool* const&, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:164:12 (libkrpc.so+0xf4205) #3 kudu::rpc::ServicePool::Init(int) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5 (libkrpc.so+0xf2b16) #4 kudu::RpcServer::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/rpc_server.cc:157:3 (libserver_process.so+0x7a799) #5 kudu::server::ServerBase::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/server_base.cc:620:23 (libserver_process.so+0x81aef) #6 kudu::master::Master::StartAsync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master.cc:186:3 (libmaster.so+0x160ddf) #7 kudu::master::MiniMaster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/mini_master.cc:94:3 (libmaster.so+0x17e254) #8 kudu::cluster::InternalMiniCluster::StartMasters() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:160:5 (libmini_cluster.so+0x409d3) #9 kudu::cluster::InternalMiniCluster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:99:3 (libmini_cluster.so+0x404ad) #10 kudu::cluster::InternalMiniCluster::StartSync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:180:3 (libmini_cluster.so+0x41984) #11 kudu::client::ClientTest_TestBlockScannerHijackingAttempts_Test::TestBody() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/client-test.cc:5884:3 (client-test+0x5c89db) #12 void testing::internal::HandleSehExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x5524f) #13 void testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x5524f) #14 testing::Test::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2474:5 (libgmock.so+0x345f8) #15 testing::TestInfo::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2656:11 (libgmock.so+0x3588c) #16 testing::TestCase::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2774:28 (libgmock.so+0x36366) #17 testing::internal::UnitTestImpl::RunAllTests() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4649:43 (libgmock.so+0x4265a) #18 bool testing::internal::HandleSehExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x561bf) #19 bool testing::internal::HandleExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x561bf) #20 testing::UnitTest::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4257:10 (libgmock.so+0x41f42) #21 RUN_ALL_TESTS() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2233:46 (libkudu_test_main.so+0x34db) #22 main /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:106:13 (libkudu_test_main.so+0x2b66) SUMMARY: ThreadSanitizer: data race /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:119 in operator delete(void*) ================== ================== WARNING: ThreadSanitizer: data race (pid=1119) Write of size 8 at 0x7b0c001139e8 by thread T164 (mutexes: read M470198777455412040, write M668920231272409128): #0 operator delete(void*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:119 (client-test+0x5550e1) #1 __libcpp_deallocate /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/new:236:3 (libc++.so.1+0xd7b20) #2 deallocate /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/memory:1796 (libc++.so.1+0xd7b20) #3 deallocate /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/memory:1555 (libc++.so.1+0xd7b20) #4 std::__1::basic_string, std::__1::allocator >::~basic_string() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/libcxx/include/string:1941 (libc++.so.1+0xd7b20) #5 google::protobuf::internal::ArenaStringPtr::DestroyNoArena(std::__1::basic_string, std::__1::allocator > const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/arenastring.h:258:7 (libmaster_proto.so+0xd17a2) #6 kudu::consensus::RaftPeerPB::SharedDtor() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1039:19 (libconsensus_metadata_proto.so+0x188d3) #7 kudu::consensus::RaftPeerPB::~RaftPeerPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1035:3 (libconsensus_metadata_proto.so+0x18873) #8 kudu::consensus::RaftPeerPB::~RaftPeerPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1033:27 (libconsensus_metadata_proto.so+0x18999) #9 google::protobuf::internal::GenericTypeHandler::Delete(kudu::consensus::RaftPeerPB*, google::protobuf::Arena*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/repeated_field.h:615:7 (libconsensus.so+0xbf5d0) #10 void google::protobuf::internal::RepeatedPtrFieldBase::Destroy::TypeHandler>() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/repeated_field.h:1429:7 (libconsensus.so+0xbf559) #11 google::protobuf::RepeatedPtrField::~RepeatedPtrField() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/google/protobuf/repeated_field.h:1892:3 (libconsensus.so+0xbe75a) #12 kudu::consensus::RaftConfigPB::~RaftConfigPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1699:1 (libconsensus_metadata_proto.so+0x19d14) #13 kudu::consensus::RaftConfigPB::~RaftConfigPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:1696:31 (libconsensus_metadata_proto.so+0x19d69) #14 kudu::consensus::ConsensusStatePB::SharedDtor() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:2189:44 (libconsensus_metadata_proto.so+0x1ae20) #15 kudu::consensus::ConsensusStatePB::~ConsensusStatePB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:2184:3 (libconsensus_metadata_proto.so+0x1ad93) #16 kudu::consensus::ConsensusStatePB::~ConsensusStatePB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/consensus/metadata.pb.cc:2182:39 (libconsensus_metadata_proto.so+0x1ae89) #17 kudu::master::SysTabletsEntryPB::SharedDtor() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.pb.cc:3181:44 (libmaster_proto.so+0x922b3) #18 kudu::master::SysTabletsEntryPB::~SysTabletsEntryPB() /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.pb.cc:3173:3 (libmaster_proto.so+0x921f3) #19 kudu::master::PersistentTabletInfo::~PersistentTabletInfo() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.h:114:8 (libmaster.so+0x13f919) #20 std::__1::default_delete::operator()(kudu::master::PersistentTabletInfo*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/memory:2285:5 (libmaster.so+0x116ea3) #21 std::__1::unique_ptr >::reset(kudu::master::PersistentTabletInfo*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/memory:2598 (libmaster.so+0x116ea3) #22 kudu::CowObject::CommitMutation() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/cow_object.h:93 (libmaster.so+0x116ea3) #23 kudu::CowGroupLock, std::__1::allocator >, kudu::master::PersistentTabletInfo>::Commit() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/cow_object.h:386:17 (libmaster.so+0x119031) #24 kudu::master::CatalogManager::ProcessTabletReport(kudu::master::TSDescriptor*, kudu::master::TabletReportPB const&, kudu::master::TabletReportUpdatesPB*, kudu::rpc::RpcContext*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.cc:4124:16 (libmaster.so+0x106ece) #25 kudu::master::MasterServiceImpl::TSHeartbeat(kudu::master::TSHeartbeatRequestPB const*, kudu::master::TSHeartbeatResponsePB*, kudu::rpc::RpcContext*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master_service.cc:257:44 (libmaster.so+0x1776c5) #26 kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_1::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.service.cc:145:13 (libmaster_proto.so+0xf21b4) #27 _ZNSt3__18__invokeIRZN4kudu6master15MasterServiceIfC1ERK13scoped_refptrINS1_12MetricEntityEERKS4_INS1_3rpc13ResultTrackerEEE3$_1JPKN6google8protobuf7MessageEPSI_PNS9_10RpcContextEEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSO_DpOSP_ /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:4482:1 (libmaster_proto.so+0xf2141) #28 void std::__1::__invoke_void_return_wrapper::__call const&, scoped_refptr const&)::$_1&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*>(kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_1&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/__functional_base:349 (libmaster_proto.so+0xf2141) #29 std::__1::__function::__func const&, scoped_refptr const&)::$_1, std::__1::allocator const&, scoped_refptr const&)::$_1>, void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1562:12 (libmaster_proto.so+0xf2054) #30 std::__1::function::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1916:12 (libkrpc.so+0xf2069) #31 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:139:3 (libkrpc.so+0xf1a05) #32 kudu::rpc::ServicePool::RunThread() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:225:15 (libkrpc.so+0xf2f0c) #33 boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/mem_fn_template.hpp:49:29 (libkrpc.so+0xf4e19) #34 void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:259:9 (libkrpc.so+0xf4d6a) #35 boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:1222:16 (libkrpc.so+0xf4cf3) #36 boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:159:11 (libkrpc.so+0xf4ae9) #37 boost::function0::operator()() const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:770:14 (libkrpc.so+0xb8411) #38 kudu::Thread::SuperviseThread(void*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:657:3 (libkudu_util.so+0x1e17d4) Previous read of size 8 at 0x7b0c001139e8 by thread T58 (mutexes: read M470198777455412040): #0 strings::memeq(void const*, void const*, unsigned long) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/strings/fastmem.h:47:44 (client-test+0x5ee83c) #1 operator==(StringPiece, StringPiece) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/strings/stringpiece.h:314:7 (client-test+0x5fb371) #2 std::__1::equal_to::operator()(StringPiece const&, StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:670:21 (libmaster.so+0x1387d7) #3 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::KeyInfo::equals(StringPiece const&, StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:1287 (libmaster.so+0x1387d7) #4 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::equals(StringPiece const&, StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:1299:21 (libmaster.so+0x13864d) #5 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::find_position(StringPiece const&) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:879:18 (libmaster.so+0x14ab8b) #6 google::dense_hashtable, StringPiece, GoodFastHash, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SelectKey, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::SetKey, std::__1::equal_to, google::libc_allocator_with_realloc > >::find(StringPiece const&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:892:43 (libmaster.so+0x14a964) #7 google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::find(StringPiece const&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/common/include/sparsehash/dense_hash_map:249:51 (libmaster.so+0x14a7f9) #8 std::__1::pair, std::__1::equal_to, google::libc_allocator_with_realloc > >::mapped_type* const, bool> ComputeIfAbsentReturnAbsense, std::__1::equal_to, google::libc_allocator_with_realloc > >, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16>(google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >*, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::key_type const&, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/map-util.h:883:53 (libmaster.so+0x11080a) #9 google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::mapped_type* const ComputeIfAbsent, std::__1::equal_to, google::libc_allocator_with_realloc > >, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16>(google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >*, google::dense_hash_map, std::__1::equal_to, google::libc_allocator_with_realloc > >::key_type const&, kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*)::$_16) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/map-util.h:906:10 (libmaster.so+0x10abb0) #10 kudu::master::CatalogManager::BuildLocationsForTablet(scoped_refptr const&, kudu::master::ReplicaTypeFilter, kudu::master::TabletLocationsPB*, kudu::master::CatalogManager::TSInfosDict*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.cc:4680:18 (libmaster.so+0x10a7aa) #11 kudu::master::CatalogManager::GetTableLocations(kudu::master::GetTableLocationsRequestPB const*, kudu::master::GetTableLocationsResponsePB*, boost::optional, std::__1::allocator > const&>) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/catalog_manager.cc:4855:16 (libmaster.so+0x10bf48) #12 kudu::master::MasterServiceImpl::GetTableLocations(kudu::master::GetTableLocationsRequestPB const*, kudu::master::GetTableLocationsResponsePB*, kudu::rpc::RpcContext*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master_service.cc:424:42 (libmaster.so+0x178b65) #13 kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_19::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/build/tsan/src/kudu/master/master.service.cc:325:13 (libmaster_proto.so+0xf61d7) #14 _ZNSt3__18__invokeIRZN4kudu6master15MasterServiceIfC1ERK13scoped_refptrINS1_12MetricEntityEERKS4_INS1_3rpc13ResultTrackerEEE4$_19JPKN6google8protobuf7MessageEPSI_PNS9_10RpcContextEEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSO_DpOSP_ /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:4482:1 (libmaster_proto.so+0xf6161) #15 void std::__1::__invoke_void_return_wrapper::__call const&, scoped_refptr const&)::$_19&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*>(kudu::master::MasterServiceIf::MasterServiceIf(scoped_refptr const&, scoped_refptr const&)::$_19&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/__functional_base:349 (libmaster_proto.so+0xf6161) #16 std::__1::__function::__func const&, scoped_refptr const&)::$_19, std::__1::allocator const&, scoped_refptr const&)::$_19>, void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1562:12 (libmaster_proto.so+0xf6074) #17 std::__1::function::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1916:12 (libkrpc.so+0xf2069) #18 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:139:3 (libkrpc.so+0xf1a05) #19 kudu::rpc::ServicePool::RunThread() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:225:15 (libkrpc.so+0xf2f0c) #20 boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/mem_fn_template.hpp:49:29 (libkrpc.so+0xf4e19) #21 void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:259:9 (libkrpc.so+0xf4d6a) #22 boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/bind/bind.hpp:1222:16 (libkrpc.so+0xf4cf3) #23 boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:159:11 (libkrpc.so+0xf4ae9) #24 boost::function0::operator()() const /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/boost/function/function_template.hpp:770:14 (libkrpc.so+0xb8411) #25 kudu::Thread::SuperviseThread(void*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:657:3 (libkudu_util.so+0x1e17d4) Mutex M470198777455412040 is already destroyed. Mutex M668920231272409128 is already destroyed. Thread T164 'rpc worker-3359' (tid=3359, running) created by main thread at: #0 pthread_create /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:992 (client-test+0x4e762b) #1 kudu::Thread::StartThread(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, boost::function const&, unsigned long, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:601:15 (libkudu_util.so+0x1e0fbb) #2 kudu::Status kudu::Thread::Create(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, void (kudu::rpc::ServicePool::* const&)(), kudu::rpc::ServicePool* const&, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:164:12 (libkrpc.so+0xf4205) #3 kudu::rpc::ServicePool::Init(int) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5 (libkrpc.so+0xf2b16) #4 kudu::RpcServer::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/rpc_server.cc:157:3 (libserver_process.so+0x7a799) #5 kudu::server::ServerBase::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/server_base.cc:620:23 (libserver_process.so+0x81aef) #6 kudu::master::Master::StartAsync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master.cc:186:3 (libmaster.so+0x160ddf) #7 kudu::master::MiniMaster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/mini_master.cc:94:3 (libmaster.so+0x17e254) #8 kudu::cluster::InternalMiniCluster::StartMasters() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:160:5 (libmini_cluster.so+0x409d3) #9 kudu::cluster::InternalMiniCluster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:99:3 (libmini_cluster.so+0x404ad) #10 kudu::cluster::InternalMiniCluster::StartSync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:180:3 (libmini_cluster.so+0x41984) #11 kudu::client::ClientTest_TestBlockScannerHijackingAttempts_Test::TestBody() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/client-test.cc:5884:3 (client-test+0x5c89db) #12 void testing::internal::HandleSehExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x5524f) #13 void testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x5524f) #14 testing::Test::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2474:5 (libgmock.so+0x345f8) #15 testing::TestInfo::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2656:11 (libgmock.so+0x3588c) #16 testing::TestCase::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2774:28 (libgmock.so+0x36366) #17 testing::internal::UnitTestImpl::RunAllTests() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4649:43 (libgmock.so+0x4265a) #18 bool testing::internal::HandleSehExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x561bf) #19 bool testing::internal::HandleExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x561bf) #20 testing::UnitTest::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4257:10 (libgmock.so+0x41f42) #21 RUN_ALL_TESTS() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2233:46 (libkudu_test_main.so+0x34db) #22 main /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:106:13 (libkudu_test_main.so+0x2b66) Thread T58 'rpc worker-3353' (tid=3353, running) created by main thread at: #0 pthread_create /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:992 (client-test+0x4e762b) #1 kudu::Thread::StartThread(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, boost::function const&, unsigned long, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:601:15 (libkudu_util.so+0x1e0fbb) #2 kudu::Status kudu::Thread::Create(std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_string, std::__1::allocator > const&, void (kudu::rpc::ServicePool::* const&)(), kudu::rpc::ServicePool* const&, scoped_refptr*) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:164:12 (libkrpc.so+0xf4205) #3 kudu::rpc::ServicePool::Init(int) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5 (libkrpc.so+0xf2b16) #4 kudu::RpcServer::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/rpc_server.cc:157:3 (libserver_process.so+0x7a799) #5 kudu::server::ServerBase::RegisterService(gscoped_ptr >) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/server_base.cc:620:23 (libserver_process.so+0x81aef) #6 kudu::master::Master::StartAsync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/master.cc:186:3 (libmaster.so+0x160ddf) #7 kudu::master::MiniMaster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/mini_master.cc:94:3 (libmaster.so+0x17e254) #8 kudu::cluster::InternalMiniCluster::StartMasters() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:160:5 (libmini_cluster.so+0x409d3) #9 kudu::cluster::InternalMiniCluster::Start() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:99:3 (libmini_cluster.so+0x404ad) #10 kudu::cluster::InternalMiniCluster::StartSync() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:180:3 (libmini_cluster.so+0x41984) #11 kudu::client::ClientTest_TestBlockScannerHijackingAttempts_Test::TestBody() /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/client-test.cc:5884:3 (client-test+0x5c89db) #12 void testing::internal::HandleSehExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x5524f) #13 void testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x5524f) #14 testing::Test::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2474:5 (libgmock.so+0x345f8) #15 testing::TestInfo::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2656:11 (libgmock.so+0x3588c) #16 testing::TestCase::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2774:28 (libgmock.so+0x36366) #17 testing::internal::UnitTestImpl::RunAllTests() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4649:43 (libgmock.so+0x4265a) #18 bool testing::internal::HandleSehExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x561bf) #19 bool testing::internal::HandleExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 (libgmock.so+0x561bf) #20 testing::UnitTest::Run() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4257:10 (libgmock.so+0x41f42) #21 RUN_ALL_TESTS() /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2233:46 (libkudu_test_main.so+0x34db) #22 main /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:106:13 (libkudu_test_main.so+0x2b66) SUMMARY: ThreadSanitizer: data race /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:119 in operator delete(void*) ================== I0609 06:57:00.368192 3359 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45098 I0609 06:57:00.380805 3474 heartbeater.cc:446] Master 127.1.23.254:46755 was elected leader, sending a full tablet report... I0609 06:57:00.693481 1119 client-test.cc:5928] Attempting to extract data from fault-tolerant scan 759595021c904d8590de733fc1c17d64 as bad-guy I0609 06:57:00.694514 3423 tablet_service.cc:2505] Scan: Not authorized: User bad-guy doesn't own scanner 759595021c904d8590de733fc1c17d64: call sequence id=1, remote={username='bad-guy'} at 127.0.0.1:38456 W0609 06:57:00.695302 1119 client.cc:1606] Scan on tablet server 37c1ed11cd8c4d7287c87baa7cb3db4d (127.1.23.193:45151) with Scanner { table: client-testtb, tablet: 49ea677949a8499480e011d8481e2c6a, projection: ( 10:key INT32 NOT NULL, 11:int_val INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, PRIMARY KEY (key) ), scan_spec: } failed: Remote error: Not authorized: User bad-guy doesn't own scanner 759595021c904d8590de733fc1c17d64 I0609 06:57:00.696890 3423 tablet_service.cc:2505] Scan: Not authorized: User bad-guy doesn't own scanner 759595021c904d8590de733fc1c17d64: call sequence id=2, remote={username='bad-guy'} at 127.0.0.1:38456 W0609 06:57:00.697799 3491 client.cc:1440] Couldn't close scanner 759595021c904d8590de733fc1c17d64: Remote error: Not authorized: User bad-guy doesn't own scanner 759595021c904d8590de733fc1c17d64 I0609 06:57:00.704797 1119 client-test.cc:5928] Attempting to extract data from non-fault-tolerant scan ce947e01aecb43478689acf90f6f136a as bad-guy I0609 06:57:00.705849 3424 tablet_service.cc:2505] Scan: Not authorized: User bad-guy doesn't own scanner ce947e01aecb43478689acf90f6f136a: call sequence id=1, remote={username='bad-guy'} at 127.0.0.1:38456 W0609 06:57:00.706638 1119 client.cc:1606] Scan on tablet server 37c1ed11cd8c4d7287c87baa7cb3db4d (127.1.23.193:45151) with Scanner { table: client-testtb, tablet: 49ea677949a8499480e011d8481e2c6a, projection: ( 10:key INT32 NOT NULL, 11:int_val INT32 NOT NULL, 12:string_val STRING NULLABLE, 13:non_null_with_default INT32 NOT NULL, PRIMARY KEY (key) ), scan_spec: } failed: Remote error: Not authorized: User bad-guy doesn't own scanner ce947e01aecb43478689acf90f6f136a I0609 06:57:00.708062 3424 tablet_service.cc:2505] Scan: Not authorized: User bad-guy doesn't own scanner ce947e01aecb43478689acf90f6f136a: call sequence id=2, remote={username='bad-guy'} at 127.0.0.1:38456 W0609 06:57:00.709197 3491 client.cc:1440] Couldn't close scanner ce947e01aecb43478689acf90f6f136a: Service unavailable: reactor is shutting down (error 108) W0609 06:57:00.710206 3404 connection.cc:647] server connection from 127.0.0.1:38456 recv error: Network error: recv error from 0.0.0.0:0: Transport endpoint is not connected (error 107) I0609 06:57:00.713456 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:57:00.735929 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:57:00.736153 1119 tablet_replica.cc:265] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d: stopping tablet replica I0609 06:57:00.736467 1119 raft_consensus.cc:2147] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 2 LEADER]: Raft consensus shutting down. I0609 06:57:00.736706 1119 raft_consensus.cc:2174] T 5964356d13164391b488b81a09e4f9b0 P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 2 FOLLOWER]: Raft consensus is shut down! I0609 06:57:00.739617 1119 tablet_replica.cc:265] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d: stopping tablet replica I0609 06:57:00.739874 1119 raft_consensus.cc:2147] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 2 LEADER]: Raft consensus shutting down. I0609 06:57:00.740084 1119 raft_consensus.cc:2174] T 49ea677949a8499480e011d8481e2c6a P 37c1ed11cd8c4d7287c87baa7cb3db4d [term 2 FOLLOWER]: Raft consensus is shut down! I0609 06:57:00.757560 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:57:00.757696 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:57:00.757938 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:57:00.764744 1119 master.cc:247] Master@127.1.23.254:46755 shutting down... I0609 06:57:00.779600 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 2 LEADER]: Raft consensus shutting down. I0609 06:57:00.779842 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f [term 2 FOLLOWER]: Raft consensus is shut down! I0609 06:57:00.780020 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 79487940486d4430a7133dacf260d00f: stopping tablet replica I0609 06:57:00.796391 1119 master.cc:258] Master@127.1.23.254:46755 shutdown complete. [ OK ] ClientTest.TestBlockScannerHijackingAttempts (1468 ms) [----------] 14 tests from ClientTest (16097 ms total) [----------] 1 test from Params/ScanMultiTabletParamTest [ RUN ] Params/ScanMultiTabletParamTest.Test/1 I0609 06:57:00.810111 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:38801 I0609 06:57:00.821578 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 97016us I0609 06:57:00.821835 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:57:00.821951 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:57:00.822122 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:57:00.822223 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:57:00.823025 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.Params_ScanMultiTabletParamTest.Test_1.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "2eebfc224be545298d093ed3424ee92a" format_stamp: "Formatted at 2019-06-09 06:57:00 on dist-test-slave-dist-test-slave-9h7f" I0609 06:57:00.831362 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.010s sys 0.000s I0609 06:57:00.832577 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:57:00.839401 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s I0609 06:57:00.839530 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.Params_ScanMultiTabletParamTest.Test_1.1560063404629499-1119/minicluster-data/master-0-root uuid: "2eebfc224be545298d093ed3424ee92a" format_stamp: "Formatted at 2019-06-09 06:57:00 on dist-test-slave-dist-test-slave-9h7f" I0609 06:57:00.839726 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.Params_ScanMultiTabletParamTest.Test_1.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.Params_ScanMultiTabletParamTest.Test_1.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.Params_ScanMultiTabletParamTest.Test_1.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:57:00.920264 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:38801 I0609 06:57:00.921977 1119 webserver.cc:374] Webserver started at http://127.1.23.254:33889/ using document root and password file I0609 06:57:00.923779 3550 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:57:00.931628 3550 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a: Bootstrap starting. I0609 06:57:00.933701 3550 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a: No blocks or log segments found. Creating new log. I0609 06:57:00.936439 3550 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a: No bootstrap required, opened a new log I0609 06:57:00.937659 3550 raft_consensus.cc:340] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eebfc224be545298d093ed3424ee92a" member_type: VOTER } I0609 06:57:00.937990 3550 raft_consensus.cc:366] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:57:00.938225 3550 raft_consensus.cc:705] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eebfc224be545298d093ed3424ee92a, State: Initialized, Role: FOLLOWER I0609 06:57:00.938632 3550 consensus_queue.cc:229] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eebfc224be545298d093ed3424ee92a" member_type: VOTER } I0609 06:57:00.938992 3550 raft_consensus.cc:380] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:00.939193 3550 raft_consensus.cc:472] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:00.939458 3550 raft_consensus.cc:2890] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [term 0 FOLLOWER]: Advancing to term 1 I0609 06:57:00.943303 3550 raft_consensus.cc:494] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eebfc224be545298d093ed3424ee92a" member_type: VOTER } I0609 06:57:00.943689 3550 leader_election.cc:310] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2eebfc224be545298d093ed3424ee92a; no voters: I0609 06:57:00.944698 3550 leader_election.cc:296] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:57:00.944892 3552 raft_consensus.cc:2644] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:57:00.946198 3552 raft_consensus.cc:667] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [term 1 LEADER]: Becoming Leader. State: Replica: 2eebfc224be545298d093ed3424ee92a, State: Running, Role: LEADER I0609 06:57:00.946630 3552 consensus_queue.cc:206] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eebfc224be545298d093ed3424ee92a" member_type: VOTER } I0609 06:57:00.949363 3550 sys_catalog.cc:338] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "2eebfc224be545298d093ed3424ee92a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eebfc224be545298d093ed3424ee92a" member_type: VOTER } } I0609 06:57:00.950028 3550 sys_catalog.cc:341] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [sys.catalog]: This master's current role is: LEADER I0609 06:57:00.951120 3554 sys_catalog.cc:338] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2eebfc224be545298d093ed3424ee92a. Latest consensus state: current_term: 1 leader_uuid: "2eebfc224be545298d093ed3424ee92a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eebfc224be545298d093ed3424ee92a" member_type: VOTER } } I0609 06:57:00.951495 3554 sys_catalog.cc:341] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [sys.catalog]: This master's current role is: LEADER I0609 06:57:00.952419 3553 sys_catalog.cc:338] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2eebfc224be545298d093ed3424ee92a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eebfc224be545298d093ed3424ee92a" member_type: VOTER } } I0609 06:57:00.952646 3550 sys_catalog.cc:426] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [sys.catalog]: configured and running, proceeding with master startup. I0609 06:57:00.952807 3553 sys_catalog.cc:341] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [sys.catalog]: This master's current role is: LEADER I0609 06:57:00.953392 3558 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:57:00.953920 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:57:00.956020 3558 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:57:00.983296 3558 catalog_manager.cc:951] Generated new certificate authority record I0609 06:57:00.984689 3558 catalog_manager.cc:1076] Loading token signing keys... I0609 06:57:00.993388 3558 catalog_manager.cc:4293] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a: Generated new TSK 0 I0609 06:57:01.031855 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 97516us I0609 06:57:01.032155 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:57:01.032308 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:57:01.032526 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:57:01.032642 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:57:01.033607 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.Params_ScanMultiTabletParamTest.Test_1.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" format_stamp: "Formatted at 2019-06-09 06:57:01 on dist-test-slave-dist-test-slave-9h7f" I0609 06:57:01.042368 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.008s user 0.006s sys 0.002s I0609 06:57:01.043721 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:57:01.050640 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:57:01.050801 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.Params_ScanMultiTabletParamTest.Test_1.1560063404629499-1119/minicluster-data/ts-0-root uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" format_stamp: "Formatted at 2019-06-09 06:57:01 on dist-test-slave-dist-test-slave-9h7f" I0609 06:57:01.051031 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.Params_ScanMultiTabletParamTest.Test_1.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.Params_ScanMultiTabletParamTest.Test_1.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.Params_ScanMultiTabletParamTest.Test_1.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:57:01.079890 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:57:01.080024 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:57:01.120831 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:45235 I0609 06:57:01.122432 1119 webserver.cc:374] Webserver started at http://127.1.23.193:39997/ using document root and password file I0609 06:57:01.134776 3625 heartbeater.cc:324] Connected to a master server at 127.1.23.254:38801 I0609 06:57:01.135028 3625 heartbeater.cc:405] Registering TS with master... I0609 06:57:01.135396 3625 heartbeater.cc:454] Master 127.1.23.254:38801 requested a full tablet report, sending... I0609 06:57:01.136795 3517 ts_manager.cc:161] Registered new tserver with Master: 2a1f0c0920a04df79adefd68e0f1d2b8 (127.1.23.193:45235) I0609 06:57:01.136912 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.012926727s I0609 06:57:01.138595 3517 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37000 I0609 06:57:01.151638 3625 heartbeater.cc:446] Master 127.1.23.254:38801 was elected leader, sending a full tablet report... I0609 06:57:01.154644 3517 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37002: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:57:01.182544 3592 tablet_service.cc:1016] Processing CreateTablet for tablet e00972ff4d784272b146ab2694ee9019 (table=client-testtb [id=7d43228a03824db8be3c7e9eea4f646d]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:57:01.182766 3591 tablet_service.cc:1016] Processing CreateTablet for tablet 703faee379e140249eac87642c0cc3b9 (table=client-testtb [id=7d43228a03824db8be3c7e9eea4f646d]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:57:01.183303 3592 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet e00972ff4d784272b146ab2694ee9019. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:57:01.183678 3591 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 703faee379e140249eac87642c0cc3b9. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:57:01.193387 3637 tablet_bootstrap.cc:439] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8: Bootstrap starting. I0609 06:57:01.195726 3637 tablet_bootstrap.cc:590] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8: No blocks or log segments found. Creating new log. I0609 06:57:01.199069 3637 tablet_bootstrap.cc:439] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8: No bootstrap required, opened a new log I0609 06:57:01.199391 3637 ts_tablet_manager.cc:1054] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s I0609 06:57:01.200475 3637 raft_consensus.cc:340] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.200806 3637 raft_consensus.cc:366] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:57:01.200956 3637 raft_consensus.cc:705] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Initialized, Role: FOLLOWER I0609 06:57:01.201462 3637 consensus_queue.cc:229] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.201895 3637 raft_consensus.cc:380] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:01.202148 3637 raft_consensus.cc:472] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:01.202364 3637 raft_consensus.cc:2890] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:57:01.206212 3637 raft_consensus.cc:494] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.206614 3637 leader_election.cc:310] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a1f0c0920a04df79adefd68e0f1d2b8; no voters: I0609 06:57:01.207880 3637 leader_election.cc:296] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:57:01.208123 3639 raft_consensus.cc:2644] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:57:01.210182 3637 ts_tablet_manager.cc:1082] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent starting tablet: real 0.011s user 0.008s sys 0.003s I0609 06:57:01.210395 3639 raft_consensus.cc:667] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Becoming Leader. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Running, Role: LEADER I0609 06:57:01.210914 3637 tablet_bootstrap.cc:439] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8: Bootstrap starting. I0609 06:57:01.210925 3639 consensus_queue.cc:206] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.214107 3637 tablet_bootstrap.cc:590] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8: No blocks or log segments found. Creating new log. I0609 06:57:01.215744 3516 catalog_manager.cc:3989] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 reported cstate change: term changed from 0 to 1, leader changed from to 2a1f0c0920a04df79adefd68e0f1d2b8 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } health_report { overall_health: HEALTHY } } } I0609 06:57:01.219331 3637 tablet_bootstrap.cc:439] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8: No bootstrap required, opened a new log I0609 06:57:01.219614 3637 ts_tablet_manager.cc:1054] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s I0609 06:57:01.220799 3637 raft_consensus.cc:340] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.221127 3637 raft_consensus.cc:366] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:57:01.221334 3637 raft_consensus.cc:705] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Initialized, Role: FOLLOWER I0609 06:57:01.221771 3637 consensus_queue.cc:229] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.222296 3637 raft_consensus.cc:380] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:01.222534 3637 raft_consensus.cc:472] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:01.222746 3637 raft_consensus.cc:2890] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:57:01.226577 3637 raft_consensus.cc:494] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.227085 3637 leader_election.cc:310] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a1f0c0920a04df79adefd68e0f1d2b8; no voters: I0609 06:57:01.227543 3637 leader_election.cc:296] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:57:01.227706 3639 raft_consensus.cc:2644] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:57:01.228250 3639 raft_consensus.cc:667] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Becoming Leader. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Running, Role: LEADER I0609 06:57:01.228546 3637 ts_tablet_manager.cc:1082] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s I0609 06:57:01.228698 3639 consensus_queue.cc:206] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.232211 3516 catalog_manager.cc:3989] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 reported cstate change: term changed from 0 to 1, leader changed from to 2a1f0c0920a04df79adefd68e0f1d2b8 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } health_report { overall_health: HEALTHY } } } I0609 06:57:01.246099 3516 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37002: name: "TestScanMultiTablet" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\n\000\000\000\004\001\000\024\000\000\000\004\001\000\036\000\000\000\004\001\000(\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:57:01.262207 3592 tablet_service.cc:1016] Processing CreateTablet for tablet e27814c97ad142728984045431b44bb8 (table=TestScanMultiTablet [id=af1e878a408a468bbcb6410d0236d58f]), partition=RANGE (key) PARTITION VALUES < 10 I0609 06:57:01.262615 3591 tablet_service.cc:1016] Processing CreateTablet for tablet c51a1b626caf42ec94229bef431c08ff (table=TestScanMultiTablet [id=af1e878a408a468bbcb6410d0236d58f]), partition=RANGE (key) PARTITION 10 <= VALUES < 20 I0609 06:57:01.263061 3592 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet e27814c97ad142728984045431b44bb8. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:57:01.263440 3591 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet c51a1b626caf42ec94229bef431c08ff. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:57:01.263216 3590 tablet_service.cc:1016] Processing CreateTablet for tablet 1bf7a6e9ef474eb89d19bc3be65bec26 (table=TestScanMultiTablet [id=af1e878a408a468bbcb6410d0236d58f]), partition=RANGE (key) PARTITION 20 <= VALUES < 30 I0609 06:57:01.264175 3590 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 1bf7a6e9ef474eb89d19bc3be65bec26. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:57:01.266088 3589 tablet_service.cc:1016] Processing CreateTablet for tablet 84f53102d5fb45449c8e0e6d4bdac917 (table=TestScanMultiTablet [id=af1e878a408a468bbcb6410d0236d58f]), partition=RANGE (key) PARTITION 30 <= VALUES < 40 I0609 06:57:01.266996 3589 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 84f53102d5fb45449c8e0e6d4bdac917. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:57:01.274168 3588 tablet_service.cc:1016] Processing CreateTablet for tablet 0fff0a7c5ecd45d58452c6e42b7395e9 (table=TestScanMultiTablet [id=af1e878a408a468bbcb6410d0236d58f]), partition=RANGE (key) PARTITION VALUES >= 40 I0609 06:57:01.275069 3588 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 0fff0a7c5ecd45d58452c6e42b7395e9. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:57:01.275272 3637 tablet_bootstrap.cc:439] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8: Bootstrap starting. I0609 06:57:01.278300 3637 tablet_bootstrap.cc:590] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8: No blocks or log segments found. Creating new log. I0609 06:57:01.285365 3637 tablet_bootstrap.cc:439] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8: No bootstrap required, opened a new log I0609 06:57:01.285763 3637 ts_tablet_manager.cc:1054] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s I0609 06:57:01.287586 3637 raft_consensus.cc:340] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.288110 3637 raft_consensus.cc:366] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:57:01.288324 3637 raft_consensus.cc:705] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Initialized, Role: FOLLOWER I0609 06:57:01.288779 3637 consensus_queue.cc:229] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.289227 3637 raft_consensus.cc:380] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:01.289458 3637 raft_consensus.cc:472] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:01.289686 3637 raft_consensus.cc:2890] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:57:01.293220 3637 raft_consensus.cc:494] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.293658 3637 leader_election.cc:310] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a1f0c0920a04df79adefd68e0f1d2b8; no voters: I0609 06:57:01.294132 3637 leader_election.cc:296] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:57:01.294275 3639 raft_consensus.cc:2644] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:57:01.294675 3639 raft_consensus.cc:667] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Becoming Leader. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Running, Role: LEADER I0609 06:57:01.295167 3637 ts_tablet_manager.cc:1082] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s I0609 06:57:01.295120 3639 consensus_queue.cc:206] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.295572 3637 tablet_bootstrap.cc:439] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8: Bootstrap starting. I0609 06:57:01.299057 3637 tablet_bootstrap.cc:590] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8: No blocks or log segments found. Creating new log. I0609 06:57:01.299111 3516 catalog_manager.cc:3989] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 reported cstate change: term changed from 0 to 1, leader changed from to 2a1f0c0920a04df79adefd68e0f1d2b8 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } health_report { overall_health: HEALTHY } } } I0609 06:57:01.304797 3637 tablet_bootstrap.cc:439] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8: No bootstrap required, opened a new log I0609 06:57:01.305173 3637 ts_tablet_manager.cc:1054] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.004s I0609 06:57:01.306550 3637 raft_consensus.cc:340] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.308229 3637 raft_consensus.cc:366] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:57:01.308472 3637 raft_consensus.cc:705] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Initialized, Role: FOLLOWER I0609 06:57:01.308959 3637 consensus_queue.cc:229] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.309442 3637 raft_consensus.cc:380] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:01.309621 3637 raft_consensus.cc:472] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:01.309826 3637 raft_consensus.cc:2890] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:57:01.313753 3637 raft_consensus.cc:494] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.314133 3637 leader_election.cc:310] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a1f0c0920a04df79adefd68e0f1d2b8; no voters: I0609 06:57:01.314558 3637 leader_election.cc:296] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:57:01.314656 3639 raft_consensus.cc:2644] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:57:01.315062 3639 raft_consensus.cc:667] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Becoming Leader. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Running, Role: LEADER I0609 06:57:01.315479 3637 ts_tablet_manager.cc:1082] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s I0609 06:57:01.315527 3639 consensus_queue.cc:206] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.316007 3637 tablet_bootstrap.cc:439] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8: Bootstrap starting. I0609 06:57:01.318816 3637 tablet_bootstrap.cc:590] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8: No blocks or log segments found. Creating new log. I0609 06:57:01.319763 3517 catalog_manager.cc:3989] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 reported cstate change: term changed from 0 to 1, leader changed from to 2a1f0c0920a04df79adefd68e0f1d2b8 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } health_report { overall_health: HEALTHY } } } I0609 06:57:01.324031 3637 tablet_bootstrap.cc:439] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8: No bootstrap required, opened a new log I0609 06:57:01.324285 3637 ts_tablet_manager.cc:1054] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s I0609 06:57:01.325914 3637 raft_consensus.cc:340] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.326314 3637 raft_consensus.cc:366] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:57:01.326570 3637 raft_consensus.cc:705] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Initialized, Role: FOLLOWER I0609 06:57:01.327060 3637 consensus_queue.cc:229] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.327489 3637 raft_consensus.cc:380] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:01.327731 3637 raft_consensus.cc:472] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:01.327960 3637 raft_consensus.cc:2890] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:57:01.331277 3637 raft_consensus.cc:494] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.331670 3637 leader_election.cc:310] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a1f0c0920a04df79adefd68e0f1d2b8; no voters: I0609 06:57:01.332084 3637 leader_election.cc:296] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:57:01.332233 3639 raft_consensus.cc:2644] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:57:01.332798 3639 raft_consensus.cc:667] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Becoming Leader. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Running, Role: LEADER I0609 06:57:01.332938 3637 ts_tablet_manager.cc:1082] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s I0609 06:57:01.333689 3637 tablet_bootstrap.cc:439] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8: Bootstrap starting. I0609 06:57:01.333511 3639 consensus_queue.cc:206] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.336846 3637 tablet_bootstrap.cc:590] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8: No blocks or log segments found. Creating new log. I0609 06:57:01.340559 3517 catalog_manager.cc:3989] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 reported cstate change: term changed from 0 to 1, leader changed from to 2a1f0c0920a04df79adefd68e0f1d2b8 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } health_report { overall_health: HEALTHY } } } I0609 06:57:01.341854 3637 tablet_bootstrap.cc:439] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8: No bootstrap required, opened a new log I0609 06:57:01.342140 3637 ts_tablet_manager.cc:1054] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s I0609 06:57:01.343823 3637 raft_consensus.cc:340] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.344168 3637 raft_consensus.cc:366] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:57:01.344347 3637 raft_consensus.cc:705] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Initialized, Role: FOLLOWER I0609 06:57:01.344779 3637 consensus_queue.cc:229] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.345255 3637 raft_consensus.cc:380] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:01.345538 3637 raft_consensus.cc:472] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:01.345762 3637 raft_consensus.cc:2890] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:57:01.350237 3637 raft_consensus.cc:494] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.350637 3637 leader_election.cc:310] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a1f0c0920a04df79adefd68e0f1d2b8; no voters: I0609 06:57:01.351053 3637 leader_election.cc:296] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:57:01.351155 3639 raft_consensus.cc:2644] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:57:01.351578 3639 raft_consensus.cc:667] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Becoming Leader. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Running, Role: LEADER I0609 06:57:01.352042 3637 ts_tablet_manager.cc:1082] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s I0609 06:57:01.352056 3639 consensus_queue.cc:206] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.352558 3637 tablet_bootstrap.cc:439] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8: Bootstrap starting. I0609 06:57:01.355717 3637 tablet_bootstrap.cc:590] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8: No blocks or log segments found. Creating new log. I0609 06:57:01.355530 3517 catalog_manager.cc:3989] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 reported cstate change: term changed from 0 to 1, leader changed from to 2a1f0c0920a04df79adefd68e0f1d2b8 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } health_report { overall_health: HEALTHY } } } I0609 06:57:01.359756 3637 tablet_bootstrap.cc:439] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8: No bootstrap required, opened a new log I0609 06:57:01.360060 3637 ts_tablet_manager.cc:1054] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s I0609 06:57:01.361666 3637 raft_consensus.cc:340] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.362011 3637 raft_consensus.cc:366] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:57:01.362248 3637 raft_consensus.cc:705] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Initialized, Role: FOLLOWER I0609 06:57:01.362761 3637 consensus_queue.cc:229] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.363212 3637 raft_consensus.cc:380] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:01.363400 3637 raft_consensus.cc:472] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:01.363574 3637 raft_consensus.cc:2890] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:57:01.367182 3637 raft_consensus.cc:494] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.367558 3637 leader_election.cc:310] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a1f0c0920a04df79adefd68e0f1d2b8; no voters: I0609 06:57:01.367928 3637 leader_election.cc:296] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:57:01.368036 3639 raft_consensus.cc:2644] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:57:01.368448 3639 raft_consensus.cc:667] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Becoming Leader. State: Replica: 2a1f0c0920a04df79adefd68e0f1d2b8, State: Running, Role: LEADER I0609 06:57:01.368827 3637 ts_tablet_manager.cc:1082] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s I0609 06:57:01.368904 3639 consensus_queue.cc:206] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } } I0609 06:57:01.372489 3517 catalog_manager.cc:3989] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 reported cstate change: term changed from 0 to 1, leader changed from to 2a1f0c0920a04df79adefd68e0f1d2b8 (127.1.23.193). New cstate: current_term: 1 leader_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1f0c0920a04df79adefd68e0f1d2b8" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 45235 } health_report { overall_health: HEALTHY } } } I0609 06:57:02.008775 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:57:02.037986 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:57:02.038228 1119 tablet_replica.cc:265] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8: stopping tablet replica I0609 06:57:02.038552 1119 raft_consensus.cc:2147] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Raft consensus shutting down. I0609 06:57:02.038872 1119 raft_consensus.cc:2174] T 0fff0a7c5ecd45d58452c6e42b7395e9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:57:02.041460 1119 tablet_replica.cc:265] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8: stopping tablet replica I0609 06:57:02.041836 1119 raft_consensus.cc:2147] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Raft consensus shutting down. I0609 06:57:02.042104 1119 raft_consensus.cc:2174] T 84f53102d5fb45449c8e0e6d4bdac917 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:57:02.044354 1119 tablet_replica.cc:265] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8: stopping tablet replica I0609 06:57:02.044740 1119 raft_consensus.cc:2147] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Raft consensus shutting down. I0609 06:57:02.045006 1119 raft_consensus.cc:2174] T 703faee379e140249eac87642c0cc3b9 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:57:02.047173 1119 tablet_replica.cc:265] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8: stopping tablet replica I0609 06:57:02.047515 1119 raft_consensus.cc:2147] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Raft consensus shutting down. I0609 06:57:02.047791 1119 raft_consensus.cc:2174] T c51a1b626caf42ec94229bef431c08ff P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:57:02.050015 1119 tablet_replica.cc:265] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8: stopping tablet replica I0609 06:57:02.050338 1119 raft_consensus.cc:2147] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Raft consensus shutting down. I0609 06:57:02.050592 1119 raft_consensus.cc:2174] T 1bf7a6e9ef474eb89d19bc3be65bec26 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:57:02.052892 1119 tablet_replica.cc:265] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8: stopping tablet replica I0609 06:57:02.053194 1119 raft_consensus.cc:2147] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Raft consensus shutting down. I0609 06:57:02.053476 1119 raft_consensus.cc:2174] T e27814c97ad142728984045431b44bb8 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:57:02.055450 1119 tablet_replica.cc:265] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8: stopping tablet replica I0609 06:57:02.055737 1119 raft_consensus.cc:2147] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 LEADER]: Raft consensus shutting down. I0609 06:57:02.055989 1119 raft_consensus.cc:2174] T e00972ff4d784272b146ab2694ee9019 P 2a1f0c0920a04df79adefd68e0f1d2b8 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:57:02.080457 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:57:02.080595 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:57:02.080869 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:57:02.090041 1119 master.cc:247] Master@127.1.23.254:38801 shutting down... I0609 06:57:02.107857 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [term 1 LEADER]: Raft consensus shutting down. I0609 06:57:02.108119 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:57:02.108374 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P 2eebfc224be545298d093ed3424ee92a: stopping tablet replica I0609 06:57:02.125210 1119 master.cc:258] Master@127.1.23.254:38801 shutdown complete. [ OK ] Params/ScanMultiTabletParamTest.Test/1 (1331 ms) [----------] 1 test from Params/ScanMultiTabletParamTest (1331 ms total) [----------] 1 test from IntColEncodings/IntEncodingNullPredicatesTest [ RUN ] IntColEncodings/IntEncodingNullPredicatesTest.TestIntEncodings/1 I0609 06:57:02.141687 1119 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.1.23.254:38771 I0609 06:57:02.152938 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 98016us I0609 06:57:02.153223 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:57:02.153394 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:57:02.153584 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:57:02.153707 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:57:02.154548 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.IntColEncodings_IntEncodingNullPredicatesTest.TestIntEncodings_1.1560063404629499-1119/minicluster-data/master-0-root/instance: uuid: "e50f705491984a28ab5fc8bd217d5f89" format_stamp: "Formatted at 2019-06-09 06:57:02 on dist-test-slave-dist-test-slave-9h7f" I0609 06:57:02.162303 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s I0609 06:57:02.163561 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:57:02.169764 1119 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s I0609 06:57:02.169960 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.IntColEncodings_IntEncodingNullPredicatesTest.TestIntEncodings_1.1560063404629499-1119/minicluster-data/master-0-root uuid: "e50f705491984a28ab5fc8bd217d5f89" format_stamp: "Formatted at 2019-06-09 06:57:02 on dist-test-slave-dist-test-slave-9h7f" I0609 06:57:02.170190 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.IntColEncodings_IntEncodingNullPredicatesTest.TestIntEncodings_1.1560063404629499-1119/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.IntColEncodings_IntEncodingNullPredicatesTest.TestIntEncodings_1.1560063404629499-1119/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.IntColEncodings_IntEncodingNullPredicatesTest.TestIntEncodings_1.1560063404629499-1119/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:57:02.237951 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.254:38771 I0609 06:57:02.239532 1119 webserver.cc:374] Webserver started at http://127.1.23.254:37989/ using document root and password file I0609 06:57:02.241039 3719 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:57:02.248248 3719 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89: Bootstrap starting. I0609 06:57:02.250356 3719 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89: No blocks or log segments found. Creating new log. I0609 06:57:02.252928 3719 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89: No bootstrap required, opened a new log I0609 06:57:02.254189 3719 raft_consensus.cc:340] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50f705491984a28ab5fc8bd217d5f89" member_type: VOTER } I0609 06:57:02.254511 3719 raft_consensus.cc:366] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:57:02.254688 3719 raft_consensus.cc:705] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e50f705491984a28ab5fc8bd217d5f89, State: Initialized, Role: FOLLOWER I0609 06:57:02.255076 3719 consensus_queue.cc:229] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50f705491984a28ab5fc8bd217d5f89" member_type: VOTER } I0609 06:57:02.255445 3719 raft_consensus.cc:380] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:02.255632 3719 raft_consensus.cc:472] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:02.255839 3719 raft_consensus.cc:2890] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [term 0 FOLLOWER]: Advancing to term 1 I0609 06:57:02.259006 3719 raft_consensus.cc:494] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50f705491984a28ab5fc8bd217d5f89" member_type: VOTER } I0609 06:57:02.259382 3719 leader_election.cc:310] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e50f705491984a28ab5fc8bd217d5f89; no voters: I0609 06:57:02.260519 3721 raft_consensus.cc:2644] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:57:02.261116 3719 leader_election.cc:296] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:57:02.262205 3721 raft_consensus.cc:667] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [term 1 LEADER]: Becoming Leader. State: Replica: e50f705491984a28ab5fc8bd217d5f89, State: Running, Role: LEADER I0609 06:57:02.262607 3721 consensus_queue.cc:206] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50f705491984a28ab5fc8bd217d5f89" member_type: VOTER } I0609 06:57:02.265352 3721 sys_catalog.cc:338] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e50f705491984a28ab5fc8bd217d5f89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50f705491984a28ab5fc8bd217d5f89" member_type: VOTER } } I0609 06:57:02.265944 3721 sys_catalog.cc:341] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [sys.catalog]: This master's current role is: LEADER I0609 06:57:02.265591 3722 sys_catalog.cc:338] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e50f705491984a28ab5fc8bd217d5f89. Latest consensus state: current_term: 1 leader_uuid: "e50f705491984a28ab5fc8bd217d5f89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50f705491984a28ab5fc8bd217d5f89" member_type: VOTER } } I0609 06:57:02.266217 3722 sys_catalog.cc:341] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [sys.catalog]: This master's current role is: LEADER I0609 06:57:02.269892 3719 sys_catalog.cc:338] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "e50f705491984a28ab5fc8bd217d5f89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50f705491984a28ab5fc8bd217d5f89" member_type: VOTER } } I0609 06:57:02.270474 3719 sys_catalog.cc:341] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [sys.catalog]: This master's current role is: LEADER I0609 06:57:02.271621 3719 sys_catalog.cc:426] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [sys.catalog]: configured and running, proceeding with master startup. I0609 06:57:02.271906 3726 catalog_manager.cc:1056] Loading table and tablet metadata into memory... I0609 06:57:02.272990 1119 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0609 06:57:02.276043 3726 catalog_manager.cc:1067] Initializing Kudu internal certificate authority... I0609 06:57:02.315243 3726 catalog_manager.cc:951] Generated new certificate authority record I0609 06:57:02.317507 3726 catalog_manager.cc:1076] Loading token signing keys... I0609 06:57:02.328764 3726 catalog_manager.cc:4293] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89: Generated new TSK 0 I0609 06:57:02.350756 1119 system_ntp.cc:193] NTP initialized. Skew: 500ppm Current error: 98016us I0609 06:57:02.351104 1119 fs_manager.cc:263] Metadata directory not provided I0609 06:57:02.351267 1119 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0609 06:57:02.351475 1119 server_base.cc:440] Could not load existing FS layout: Not found: could not find a healthy instance file I0609 06:57:02.351599 1119 server_base.cc:441] Attempting to create new FS layout instead I0609 06:57:02.352499 1119 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.IntColEncodings_IntEncodingNullPredicatesTest.TestIntEncodings_1.1560063404629499-1119/minicluster-data/ts-0-root/instance: uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" format_stamp: "Formatted at 2019-06-09 06:57:02 on dist-test-slave-dist-test-slave-9h7f" I0609 06:57:02.362273 1119 fs_manager.cc:503] Time spent creating directory manager: real 0.009s user 0.010s sys 0.001s I0609 06:57:02.363620 1119 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0609 06:57:02.374161 1119 fs_manager.cc:419] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s I0609 06:57:02.374325 1119 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.IntColEncodings_IntEncodingNullPredicatesTest.TestIntEncodings_1.1560063404629499-1119/minicluster-data/ts-0-root uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" format_stamp: "Formatted at 2019-06-09 06:57:02 on dist-test-slave-dist-test-slave-9h7f" I0609 06:57:02.374590 1119 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.IntColEncodings_IntEncodingNullPredicatesTest.TestIntEncodings_1.1560063404629499-1119/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.IntColEncodings_IntEncodingNullPredicatesTest.TestIntEncodings_1.1560063404629499-1119/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskY5QUuL/test-tmp/client-test.3.IntColEncodings_IntEncodingNullPredicatesTest.TestIntEncodings_1.1560063404629499-1119/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0609 06:57:02.417145 1119 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0609 06:57:02.417377 1119 ts_tablet_manager.cc:376] Registered 0 tablets I0609 06:57:02.540693 1119 rpc_server.cc:205] RPC server started. Bound to: 127.1.23.193:36499 I0609 06:57:02.542603 1119 webserver.cc:374] Webserver started at http://127.1.23.193:43205/ using document root and password file I0609 06:57:02.556546 3794 heartbeater.cc:324] Connected to a master server at 127.1.23.254:38771 I0609 06:57:02.556792 3794 heartbeater.cc:405] Registering TS with master... I0609 06:57:02.557209 3794 heartbeater.cc:454] Master 127.1.23.254:38771 requested a full tablet report, sending... I0609 06:57:02.558635 3686 ts_manager.cc:161] Registered new tserver with Master: 9e8e91cee81e41949dc8a8c1bf5c45ce (127.1.23.193:36499) I0609 06:57:02.559502 1119 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.01499052s I0609 06:57:02.560276 3686 master_service.cc:274] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53664 I0609 06:57:02.572659 3794 heartbeater.cc:446] Master 127.1.23.254:38771 was elected leader, sending a full tablet report... I0609 06:57:02.583748 3686 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53666: name: "client-testtb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\t\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0609 06:57:02.617324 3761 tablet_service.cc:1016] Processing CreateTablet for tablet 04ee444de7ff4da5a677ff23770f71d7 (table=client-testtb [id=feee23b3888943c4b41c8e5d189ade3d]), partition=RANGE (key) PARTITION VALUES < 9 I0609 06:57:02.617879 3760 tablet_service.cc:1016] Processing CreateTablet for tablet d88e19b65a02410786c209cbb05d0d18 (table=client-testtb [id=feee23b3888943c4b41c8e5d189ade3d]), partition=RANGE (key) PARTITION VALUES >= 9 I0609 06:57:02.618232 3761 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 04ee444de7ff4da5a677ff23770f71d7. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:57:02.619383 3760 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet d88e19b65a02410786c209cbb05d0d18. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:57:02.630748 3806 tablet_bootstrap.cc:439] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce: Bootstrap starting. I0609 06:57:02.634837 3806 tablet_bootstrap.cc:590] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce: No blocks or log segments found. Creating new log. I0609 06:57:02.638203 3806 tablet_bootstrap.cc:439] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce: No bootstrap required, opened a new log I0609 06:57:02.638631 3806 ts_tablet_manager.cc:1054] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s I0609 06:57:02.640365 3806 raft_consensus.cc:340] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } } I0609 06:57:02.640954 3806 raft_consensus.cc:366] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:57:02.641189 3806 raft_consensus.cc:705] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e8e91cee81e41949dc8a8c1bf5c45ce, State: Initialized, Role: FOLLOWER I0609 06:57:02.641763 3806 consensus_queue.cc:229] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } } I0609 06:57:02.642349 3806 raft_consensus.cc:380] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:02.642575 3806 raft_consensus.cc:472] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:02.642889 3806 raft_consensus.cc:2890] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Advancing to term 1 I0609 06:57:02.648442 3806 raft_consensus.cc:494] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } } I0609 06:57:02.649019 3806 leader_election.cc:310] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e8e91cee81e41949dc8a8c1bf5c45ce; no voters: I0609 06:57:02.650490 3806 leader_election.cc:296] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:57:02.650640 3808 raft_consensus.cc:2644] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:57:02.652814 3806 ts_tablet_manager.cc:1082] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s I0609 06:57:02.652842 3808 raft_consensus.cc:667] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 LEADER]: Becoming Leader. State: Replica: 9e8e91cee81e41949dc8a8c1bf5c45ce, State: Running, Role: LEADER I0609 06:57:02.653424 3806 tablet_bootstrap.cc:439] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce: Bootstrap starting. I0609 06:57:02.653662 3808 consensus_queue.cc:206] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } } I0609 06:57:02.655732 3806 tablet_bootstrap.cc:590] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce: No blocks or log segments found. Creating new log. I0609 06:57:02.660836 3806 tablet_bootstrap.cc:439] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce: No bootstrap required, opened a new log I0609 06:57:02.661464 3806 ts_tablet_manager.cc:1054] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s I0609 06:57:02.660928 3686 catalog_manager.cc:3989] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce reported cstate change: term changed from 0 to 1, leader changed from to 9e8e91cee81e41949dc8a8c1bf5c45ce (127.1.23.193). New cstate: current_term: 1 leader_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } health_report { overall_health: HEALTHY } } } I0609 06:57:02.662881 3806 raft_consensus.cc:340] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } } I0609 06:57:02.663950 3806 raft_consensus.cc:366] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:57:02.664374 3806 raft_consensus.cc:705] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e8e91cee81e41949dc8a8c1bf5c45ce, State: Initialized, Role: FOLLOWER I0609 06:57:02.665119 3806 consensus_queue.cc:229] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } } I0609 06:57:02.665625 3806 raft_consensus.cc:380] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:02.665879 3806 raft_consensus.cc:472] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:02.666090 3806 raft_consensus.cc:2890] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Advancing to term 1 I0609 06:57:02.670315 3806 raft_consensus.cc:494] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } } I0609 06:57:02.671046 3806 leader_election.cc:310] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e8e91cee81e41949dc8a8c1bf5c45ce; no voters: I0609 06:57:02.671669 3806 leader_election.cc:296] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:57:02.671798 3808 raft_consensus.cc:2644] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:57:02.672538 3808 raft_consensus.cc:667] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 LEADER]: Becoming Leader. State: Replica: 9e8e91cee81e41949dc8a8c1bf5c45ce, State: Running, Role: LEADER I0609 06:57:02.672969 3806 ts_tablet_manager.cc:1082] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s I0609 06:57:02.673243 3808 consensus_queue.cc:206] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } } I0609 06:57:02.679395 3686 catalog_manager.cc:3989] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce reported cstate change: term changed from 0 to 1, leader changed from to 9e8e91cee81e41949dc8a8c1bf5c45ce (127.1.23.193). New cstate: current_term: 1 leader_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } health_report { overall_health: HEALTHY } } } I0609 06:57:02.690219 3685 catalog_manager.cc:1410] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53666: name: "IntEncodingNullPredicatesTestTable" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: true encoding: BIT_SHUFFLE compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { } partition_schema { range_schema { columns { name: "key" } } } I0609 06:57:02.704295 3761 tablet_service.cc:1016] Processing CreateTablet for tablet 4b7fe0f1e5ef4fbfb71cd3e3944c2894 (table=IntEncodingNullPredicatesTestTable [id=eddcb6c79fe940728fbf45be9d1972d2]), partition=RANGE (key) PARTITION UNBOUNDED I0609 06:57:02.704896 3761 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 4b7fe0f1e5ef4fbfb71cd3e3944c2894. 1 dirs total, 0 dirs full, 0 dirs failed I0609 06:57:02.714531 3806 tablet_bootstrap.cc:439] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce: Bootstrap starting. I0609 06:57:02.717393 3806 tablet_bootstrap.cc:590] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce: No blocks or log segments found. Creating new log. I0609 06:57:02.722848 3806 tablet_bootstrap.cc:439] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce: No bootstrap required, opened a new log I0609 06:57:02.723131 3806 ts_tablet_manager.cc:1054] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s I0609 06:57:02.724294 3806 raft_consensus.cc:340] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } } I0609 06:57:02.724754 3806 raft_consensus.cc:366] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0609 06:57:02.724925 3806 raft_consensus.cc:705] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e8e91cee81e41949dc8a8c1bf5c45ce, State: Initialized, Role: FOLLOWER I0609 06:57:02.727463 3806 consensus_queue.cc:229] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } } I0609 06:57:02.728055 3806 raft_consensus.cc:380] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0609 06:57:02.728261 3806 raft_consensus.cc:472] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0609 06:57:02.728540 3806 raft_consensus.cc:2890] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 0 FOLLOWER]: Advancing to term 1 I0609 06:57:02.732260 3806 raft_consensus.cc:494] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } } I0609 06:57:02.732828 3806 leader_election.cc:310] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e8e91cee81e41949dc8a8c1bf5c45ce; no voters: I0609 06:57:02.733645 3808 raft_consensus.cc:2644] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 FOLLOWER]: Leader election won for term 1 I0609 06:57:02.734053 3808 raft_consensus.cc:667] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 LEADER]: Becoming Leader. State: Replica: 9e8e91cee81e41949dc8a8c1bf5c45ce, State: Running, Role: LEADER I0609 06:57:02.734596 3808 consensus_queue.cc:206] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } } I0609 06:57:02.737782 3806 leader_election.cc:296] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [CANDIDATE]: Term 1 election: Requested vote from peers I0609 06:57:02.738950 3806 ts_tablet_manager.cc:1082] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce: Time spent starting tablet: real 0.016s user 0.010s sys 0.000s I0609 06:57:02.740092 3685 catalog_manager.cc:3989] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce reported cstate change: term changed from 0 to 1, leader changed from to 9e8e91cee81e41949dc8a8c1bf5c45ce (127.1.23.193). New cstate: current_term: 1 leader_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e8e91cee81e41949dc8a8c1bf5c45ce" member_type: VOTER last_known_addr { host: "127.1.23.193" port: 36499 } health_report { overall_health: HEALTHY } } } I0609 06:57:03.250028 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:57:03.277907 1119 ts_tablet_manager.cc:1128] Shutting down tablet manager... I0609 06:57:03.278198 1119 tablet_replica.cc:265] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce: stopping tablet replica I0609 06:57:03.278504 1119 raft_consensus.cc:2147] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 LEADER]: Raft consensus shutting down. I0609 06:57:03.278789 1119 raft_consensus.cc:2174] T 4b7fe0f1e5ef4fbfb71cd3e3944c2894 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:57:03.282244 1119 tablet_replica.cc:265] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce: stopping tablet replica I0609 06:57:03.282547 1119 raft_consensus.cc:2147] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 LEADER]: Raft consensus shutting down. I0609 06:57:03.282812 1119 raft_consensus.cc:2174] T 04ee444de7ff4da5a677ff23770f71d7 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:57:03.284899 1119 tablet_replica.cc:265] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce: stopping tablet replica I0609 06:57:03.285209 1119 raft_consensus.cc:2147] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 LEADER]: Raft consensus shutting down. I0609 06:57:03.285503 1119 raft_consensus.cc:2174] T d88e19b65a02410786c209cbb05d0d18 P 9e8e91cee81e41949dc8a8c1bf5c45ce [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:57:03.298257 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:57:03.298393 1119 tablet_server.cc:155] TabletServer shutting down... I0609 06:57:03.298648 1119 tablet_server.cc:169] TabletServer shutdown complete. I0609 06:57:03.305244 1119 master.cc:247] Master@127.1.23.254:38771 shutting down... I0609 06:57:03.324723 1119 raft_consensus.cc:2147] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [term 1 LEADER]: Raft consensus shutting down. I0609 06:57:03.325037 1119 raft_consensus.cc:2174] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89 [term 1 FOLLOWER]: Raft consensus is shut down! I0609 06:57:03.325273 1119 tablet_replica.cc:265] T 00000000000000000000000000000000 P e50f705491984a28ab5fc8bd217d5f89: stopping tablet replica I0609 06:57:03.342337 1119 master.cc:258] Master@127.1.23.254:38771 shutdown complete. [ OK ] IntColEncodings/IntEncodingNullPredicatesTest.TestIntEncodings/1 (1215 ms) [----------] 1 test from IntColEncodings/IntEncodingNullPredicatesTest (1215 ms total) [----------] Global test environment tear-down [==========] 16 tests from 3 test cases ran. (18645 ms total) [ PASSED ] 16 tests. ThreadSanitizer: reported 4 warnings