Note: This is test shard 3 of 4. [==========] Running 22 tests from 5 test cases. [----------] Global test environment set-up. [----------] 17 tests from ToolTest [ RUN ] ToolTest.TestModeHelp [ OK ] ToolTest.TestModeHelp (2420 ms) [ RUN ] ToolTest.TestFsCheckLiveServer 2020-01-07T02:52:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG) 2020-01-07T02:52:32Z Disabled control of system clock WARNING: Logging before InitGoogleLogging() is written to STDERR I0107 02:52:32.297019 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/data --block_manager=log --webserver_interface=localhost --ipki_ca_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:44531 --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --rpc_reuseport=true --builtin_ntp_servers=127.2.165.148:44297 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:32.364336 2818 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:32.364579 2818 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:32.365617 2818 flags.cc:404] Enabled experimental flag: --ipki_ca_key_size=1024 W0107 02:52:32.365640 2818 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:32.365649 2818 flags.cc:404] Enabled experimental flag: --tsk_num_rsa_bits=512 W0107 02:52:32.365661 2818 flags.cc:404] Enabled experimental flag: --rpc_reuseport=true I0107 02:52:32.367686 2818 master_main.cc:96] Master server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:44297 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/wal --ipki_ca_key_size=1024 --ipki_server_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:44531 --rpc_reuseport=true --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/data/info.pb --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync=true --heap_profile_path=/tmp/kudu-master.2818 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/logs --logbuflevel=-1 --logtostderr=true Master server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:32.376991 2818 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:32.378227 2818 hybrid_clock.cc:186] HybridClock initialized: now 1578365552378150 us; error 61 us; skew 500 ppm I0107 02:52:32.378947 2818 fs_manager.cc:263] Metadata directory not provided I0107 02:52:32.378996 2818 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:32.379257 2818 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:32.379287 2818 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:32.381613 2818 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/data/instance: uuid: "feea08e3e2fc419688b07bb173973598" format_stamp: "Formatted at 2020-01-07 02:52:32 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:32.381893 2818 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/wal/instance: uuid: "feea08e3e2fc419688b07bb173973598" format_stamp: "Formatted at 2020-01-07 02:52:32 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:32.383610 2818 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:32.389432 2818 fs_manager.cc:518] Time spent creating directory manager: real 0.007s user 0.006s sys 0.004s I0107 02:52:32.390106 2818 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:32.390630 2818 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:32.394376 2818 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:32.394517 2818 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/wal uuid: "feea08e3e2fc419688b07bb173973598" format_stamp: "Formatted at 2020-01-07 02:52:32 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:32.394635 2818 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:32.417083 2818 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:32.423316 2818 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.190:44531 I0107 02:52:32.424343 2818 webserver.cc:382] Webserver started at http://127.2.165.190:39407/ using document root and password file I0107 02:52:32.425057 2818 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/master-0/data/info.pb I0107 02:52:32.428565 2879 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:32.432736 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master as pid 2818 I0107 02:52:32.433480 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/data --block_manager=log --rpc_bind_addresses=127.2.165.129:0 --local_ip_for_outbound_sockets=127.2.165.129 --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:44531 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.2.165.148:44297 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} I0107 02:52:32.434276 2879 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598: Bootstrap starting. I0107 02:52:32.437767 2879 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598: No blocks or log segments found. Creating new log. I0107 02:52:32.438814 2879 log.cc:815] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598: Log is configured to *not* fsync() on all Append() calls I0107 02:52:32.443622 2879 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598: No bootstrap required, opened a new log I0107 02:52:32.446758 2879 raft_consensus.cc:340] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feea08e3e2fc419688b07bb173973598" member_type: VOTER } I0107 02:52:32.446851 2879 raft_consensus.cc:366] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:32.446900 2879 raft_consensus.cc:705] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feea08e3e2fc419688b07bb173973598, State: Initialized, Role: FOLLOWER I0107 02:52:32.447394 2879 consensus_queue.cc:231] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feea08e3e2fc419688b07bb173973598" member_type: VOTER } I0107 02:52:32.447459 2879 raft_consensus.cc:380] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:32.447504 2879 raft_consensus.cc:472] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:32.447577 2879 raft_consensus.cc:2918] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:32.448365 2879 raft_consensus.cc:494] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feea08e3e2fc419688b07bb173973598" member_type: VOTER } I0107 02:52:32.448807 2879 leader_election.cc:307] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: feea08e3e2fc419688b07bb173973598; no voters: I0107 02:52:32.449231 2879 leader_election.cc:293] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:32.449584 2879 sys_catalog.cc:339] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feea08e3e2fc419688b07bb173973598" member_type: VOTER } } I0107 02:52:32.449831 2882 raft_consensus.cc:2672] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:32.449991 2882 raft_consensus.cc:667] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [term 1 LEADER]: Becoming Leader. State: Replica: feea08e3e2fc419688b07bb173973598, State: Running, Role: LEADER I0107 02:52:32.450285 2882 consensus_queue.cc:208] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feea08e3e2fc419688b07bb173973598" member_type: VOTER } I0107 02:52:32.449615 2879 sys_catalog.cc:342] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [sys.catalog]: This master's current role is: FOLLOWER I0107 02:52:32.453208 2884 sys_catalog.cc:339] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader feea08e3e2fc419688b07bb173973598. Latest consensus state: current_term: 1 leader_uuid: "feea08e3e2fc419688b07bb173973598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feea08e3e2fc419688b07bb173973598" member_type: VOTER } } I0107 02:52:32.453243 2884 sys_catalog.cc:342] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [sys.catalog]: This master's current role is: LEADER I0107 02:52:32.453464 2879 sys_catalog.cc:436] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [sys.catalog]: configured and running, proceeding with master startup. I0107 02:52:32.453626 2883 sys_catalog.cc:339] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "feea08e3e2fc419688b07bb173973598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feea08e3e2fc419688b07bb173973598" member_type: VOTER } } I0107 02:52:32.453656 2883 sys_catalog.cc:342] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598 [sys.catalog]: This master's current role is: LEADER W0107 02:52:32.457945 2887 catalog_manager.cc:1158] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598: acquiring CA information for follower catalog manager: Not found: root CA entry not found W0107 02:52:32.457993 2887 catalog_manager.cc:630] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry I0107 02:52:32.458055 2886 catalog_manager.cc:1089] Loading table and tablet metadata into memory... I0107 02:52:32.459244 2886 catalog_manager.cc:1100] Initializing Kudu internal certificate authority... WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:32.490474 2880 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:32.490761 2880 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:32.491919 2880 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:32.491986 2880 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.165.129 I0107 02:52:32.493710 2880 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:44297 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.2.165.129:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:44531 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.2880 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.2.165.129 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:32.495970 2886 catalog_manager.cc:984] Generated new certificate authority record I0107 02:52:32.497439 2886 catalog_manager.cc:1109] Loading token signing keys... I0107 02:52:32.506987 2880 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:32.508374 2886 catalog_manager.cc:4455] T 00000000000000000000000000000000 P feea08e3e2fc419688b07bb173973598: Generated new TSK 0 I0107 02:52:32.508971 2886 catalog_manager.cc:1119] Initializing in-progress tserver states... I0107 02:52:32.509796 2880 hybrid_clock.cc:186] HybridClock initialized: now 1578365552509741 us; error 42 us; skew 500 ppm I0107 02:52:32.510488 2880 fs_manager.cc:263] Metadata directory not provided I0107 02:52:32.510535 2880 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:32.510805 2880 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:32.510830 2880 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:32.512976 2880 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/data/instance: uuid: "8452d387dafd474faf756fd299d25c7a" format_stamp: "Formatted at 2020-01-07 02:52:32 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:32.513264 2880 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/wal/instance: uuid: "8452d387dafd474faf756fd299d25c7a" format_stamp: "Formatted at 2020-01-07 02:52:32 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:32.514889 2880 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:32.520495 2880 fs_manager.cc:518] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s I0107 02:52:32.521420 2880 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:32.521966 2880 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:32.526646 2880 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s I0107 02:52:32.526806 2880 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/wal uuid: "8452d387dafd474faf756fd299d25c7a" format_stamp: "Formatted at 2020-01-07 02:52:32 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:32.526957 2880 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:32.596410 2880 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:32.598201 2880 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:32.598239 2880 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:32.609572 2880 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.129:36459 I0107 02:52:32.610744 2880 webserver.cc:382] Webserver started at http://127.2.165.129:32909/ using document root and password file I0107 02:52:32.611519 2880 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestFsCheckLiveServer.1578365549852848-2710/minicluster-data/ts-0/data/info.pb I0107 02:52:32.614131 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 2880 I0107 02:52:32.624608 3011 heartbeater.cc:322] Connected to a master server at 127.2.165.190:44531 I0107 02:52:32.624775 3011 heartbeater.cc:406] Registering TS with master... I0107 02:52:32.625249 3011 heartbeater.cc:455] Master 127.2.165.190:44531 requested a full tablet report, sending... I0107 02:52:32.626866 2845 ts_manager.cc:192] Registered new tserver with Master: 8452d387dafd474faf756fd299d25c7a (127.2.165.129:36459) I0107 02:52:32.628574 2845 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 127.2.165.129:37319 I0107 02:52:32.629614 2710 external_mini_cluster.cc:721] 1 TS(s) registered with all masters I0107 02:52:32.842833 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 2880 I0107 02:52:32.846314 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master with pid 2818 2020-01-07T02:52:32Z chronyd exiting [ OK ] ToolTest.TestFsCheckLiveServer (586 ms) [ RUN ] ToolTest.TestPbcTools I0107 02:52:32.867571 2710 fs_manager.cc:263] Metadata directory not provided I0107 02:52:32.867632 2710 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:32.869976 2710 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestPbcTools.1578365549852848-2710/test/instance: uuid: "33fefd586fd345cf8789fe9e069ae713" format_stamp: "Formatted at 2020-01-07 02:52:32 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:32.872184 2710 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestPbcTools.1578365549852848-2710/test/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestPbcTools.1578365549852848-2710/test/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:32.878587 2710 fs_manager.cc:518] Time spent creating directory manager: real 0.008s user 0.011s sys 0.000s I0107 02:52:32.879788 2710 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:32.880388 2710 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:32.884606 2710 fs_report.cc:352] FS layout report -------------------- wal directory: metadata directory: 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestPbcTools.1578365549852848-2710/test/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:32.884724 2710 fs_manager.cc:434] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s I0107 02:52:32.884791 2710 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestPbcTools.1578365549852848-2710/test uuid: "33fefd586fd345cf8789fe9e069ae713" format_stamp: "Formatted at 2020-01-07 02:52:32 on dist-test-slave-dist-test-slave-3kpx" [ OK ] ToolTest.TestPbcTools (1140 ms) [ RUN ] ToolTest.TestLocalReplicaDumpDataDirs I0107 02:52:34.008270 2710 fs_manager.cc:263] Metadata directory not provided I0107 02:52:34.008333 2710 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:34.008816 2710 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data0/instance: uuid: "f8858f58bd0445c4b55c4a8594a9eb92" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:34.009003 2710 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data1/instance: uuid: "f8858f58bd0445c4b55c4a8594a9eb92" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:34.009148 2710 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data2/instance: uuid: "f8858f58bd0445c4b55c4a8594a9eb92" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:34.009289 2710 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/wal/instance: uuid: "f8858f58bd0445c4b55c4a8594a9eb92" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:34.010483 2710 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data0/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data0/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:34.010550 2710 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data1/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data1/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:34.010591 2710 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data2/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data2/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:34.015027 2710 fs_manager.cc:518] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s I0107 02:52:34.015722 2710 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:34.017587 2710 fs_report.cc:352] FS layout report -------------------- wal directory: metadata directory: 3 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data0/data, /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data1/data, /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data2/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:34.017643 2710 fs_manager.cc:434] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s I0107 02:52:34.017697 2710 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data0,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data1,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/data2,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaDumpDataDirs.1578365549852848-2710/wal uuid: "f8858f58bd0445c4b55c4a8594a9eb92" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" [ OK ] ToolTest.TestLocalReplicaDumpDataDirs (139 ms) [ RUN ] ToolTest.TestLoadgenDefaultParameters 2020-01-07T02:52:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG) 2020-01-07T02:52:34Z Disabled control of system clock I0107 02:52:34.152174 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/data --block_manager=log --webserver_interface=localhost --ipki_ca_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:40613 --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --rpc_reuseport=true --builtin_ntp_servers=127.2.165.148:33753 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:34.217607 3097 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:34.217824 3097 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:34.218859 3097 flags.cc:404] Enabled experimental flag: --ipki_ca_key_size=1024 W0107 02:52:34.218881 3097 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:34.218888 3097 flags.cc:404] Enabled experimental flag: --tsk_num_rsa_bits=512 W0107 02:52:34.218897 3097 flags.cc:404] Enabled experimental flag: --rpc_reuseport=true I0107 02:52:34.220854 3097 master_main.cc:96] Master server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:33753 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/wal --ipki_ca_key_size=1024 --ipki_server_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:40613 --rpc_reuseport=true --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/data/info.pb --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync=true --heap_profile_path=/tmp/kudu-master.3097 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/logs --logbuflevel=-1 --logtostderr=true Master server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:34.230146 3097 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:34.231391 3097 hybrid_clock.cc:186] HybridClock initialized: now 1578365554231339 us; error 37 us; skew 500 ppm I0107 02:52:34.232138 3097 fs_manager.cc:263] Metadata directory not provided I0107 02:52:34.232183 3097 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:34.232437 3097 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:34.232461 3097 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:34.234817 3097 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/data/instance: uuid: "b271f566a92241febcaa7d00d4352f71" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:34.235074 3097 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/wal/instance: uuid: "b271f566a92241febcaa7d00d4352f71" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:34.236753 3097 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:34.242362 3097 fs_manager.cc:518] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s I0107 02:52:34.243150 3097 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:34.243645 3097 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:34.247689 3097 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s I0107 02:52:34.247818 3097 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/wal uuid: "b271f566a92241febcaa7d00d4352f71" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:34.247951 3097 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:34.288496 3097 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:34.294608 3097 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.190:40613 I0107 02:52:34.295694 3097 webserver.cc:382] Webserver started at http://127.2.165.190:33045/ using document root and password file I0107 02:52:34.296386 3097 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/master-0/data/info.pb I0107 02:52:34.297050 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master as pid 3097 I0107 02:52:34.297515 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/data --block_manager=log --rpc_bind_addresses=127.2.165.129:0 --local_ip_for_outbound_sockets=127.2.165.129 --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:40613 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.2.165.148:33753 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} I0107 02:52:34.299988 3158 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:34.305773 3158 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: Bootstrap starting. I0107 02:52:34.308326 3158 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: No blocks or log segments found. Creating new log. I0107 02:52:34.309317 3158 log.cc:815] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: Log is configured to *not* fsync() on all Append() calls I0107 02:52:34.310803 3158 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: No bootstrap required, opened a new log I0107 02:52:34.313812 3158 raft_consensus.cc:340] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b271f566a92241febcaa7d00d4352f71" member_type: VOTER } I0107 02:52:34.313887 3158 raft_consensus.cc:366] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:34.313931 3158 raft_consensus.cc:705] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b271f566a92241febcaa7d00d4352f71, State: Initialized, Role: FOLLOWER I0107 02:52:34.314514 3158 consensus_queue.cc:231] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b271f566a92241febcaa7d00d4352f71" member_type: VOTER } I0107 02:52:34.314786 3158 raft_consensus.cc:380] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:34.314851 3158 raft_consensus.cc:472] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:34.314954 3158 raft_consensus.cc:2918] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:34.315914 3158 raft_consensus.cc:494] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b271f566a92241febcaa7d00d4352f71" member_type: VOTER } I0107 02:52:34.316418 3158 leader_election.cc:307] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b271f566a92241febcaa7d00d4352f71; no voters: I0107 02:52:34.316884 3158 leader_election.cc:293] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:34.317224 3158 sys_catalog.cc:339] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b271f566a92241febcaa7d00d4352f71" member_type: VOTER } } I0107 02:52:34.317253 3158 sys_catalog.cc:342] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [sys.catalog]: This master's current role is: FOLLOWER I0107 02:52:34.318131 3158 sys_catalog.cc:436] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [sys.catalog]: configured and running, proceeding with master startup. I0107 02:52:34.318639 3161 raft_consensus.cc:2672] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:34.319195 3161 raft_consensus.cc:667] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [term 1 LEADER]: Becoming Leader. State: Replica: b271f566a92241febcaa7d00d4352f71, State: Running, Role: LEADER I0107 02:52:34.319530 3161 consensus_queue.cc:208] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b271f566a92241febcaa7d00d4352f71" member_type: VOTER } I0107 02:52:34.320762 3161 sys_catalog.cc:339] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b271f566a92241febcaa7d00d4352f71. Latest consensus state: current_term: 1 leader_uuid: "b271f566a92241febcaa7d00d4352f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b271f566a92241febcaa7d00d4352f71" member_type: VOTER } } I0107 02:52:34.320796 3161 sys_catalog.cc:342] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [sys.catalog]: This master's current role is: LEADER I0107 02:52:34.321432 3162 sys_catalog.cc:339] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b271f566a92241febcaa7d00d4352f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b271f566a92241febcaa7d00d4352f71" member_type: VOTER } } I0107 02:52:34.321519 3162 sys_catalog.cc:342] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71 [sys.catalog]: This master's current role is: LEADER W0107 02:52:34.324910 3163 catalog_manager.cc:1158] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: acquiring CA information for follower catalog manager: Not found: root CA entry not found W0107 02:52:34.324960 3163 catalog_manager.cc:630] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry I0107 02:52:34.325027 3166 catalog_manager.cc:1089] Loading table and tablet metadata into memory... I0107 02:52:34.326355 3166 catalog_manager.cc:1100] Initializing Kudu internal certificate authority... I0107 02:52:34.348174 3166 catalog_manager.cc:984] Generated new certificate authority record I0107 02:52:34.349712 3166 catalog_manager.cc:1109] Loading token signing keys... WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:34.356114 3159 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:34.356361 3159 flags.cc:404] Enabled unsafe flag: --never_fsync=true I0107 02:52:34.357118 3166 catalog_manager.cc:4455] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: Generated new TSK 0 W0107 02:52:34.357642 3159 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:34.357731 3159 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.165.129 I0107 02:52:34.357789 3166 catalog_manager.cc:1119] Initializing in-progress tserver states... I0107 02:52:34.359871 3159 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:33753 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.2.165.129:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:40613 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.3159 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.2.165.129 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:34.369494 3159 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:34.377005 3159 hybrid_clock.cc:186] HybridClock initialized: now 1578365554376944 us; error 50 us; skew 500 ppm I0107 02:52:34.377864 3159 fs_manager.cc:263] Metadata directory not provided I0107 02:52:34.377918 3159 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:34.378190 3159 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:34.378216 3159 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:34.380520 3159 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/data/instance: uuid: "965d933521a54274adc66bf141c4be49" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:34.380812 3159 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/wal/instance: uuid: "965d933521a54274adc66bf141c4be49" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:34.382529 3159 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:34.391993 3159 fs_manager.cc:518] Time spent creating directory manager: real 0.011s user 0.009s sys 0.000s I0107 02:52:34.392834 3159 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:34.393375 3159 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:34.397091 3159 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:34.397294 3159 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/wal uuid: "965d933521a54274adc66bf141c4be49" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:34.397441 3159 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:34.434857 3159 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:34.436187 3159 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:34.436221 3159 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:34.443940 3159 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.129:37549 I0107 02:52:34.444898 3159 webserver.cc:382] Webserver started at http://127.2.165.129:42433/ using document root and password file I0107 02:52:34.445493 3159 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/data/info.pb I0107 02:52:34.453229 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 3159 I0107 02:52:34.464334 3290 heartbeater.cc:322] Connected to a master server at 127.2.165.190:40613 I0107 02:52:34.464499 3290 heartbeater.cc:406] Registering TS with master... I0107 02:52:34.465065 3290 heartbeater.cc:455] Master 127.2.165.190:40613 requested a full tablet report, sending... I0107 02:52:34.466646 3124 ts_manager.cc:192] Registered new tserver with Master: 965d933521a54274adc66bf141c4be49 (127.2.165.129:37549) I0107 02:52:34.468297 2710 external_mini_cluster.cc:721] 1 TS(s) registered with all masters I0107 02:52:34.468400 3124 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 127.2.165.129:33119 W0107 02:52:34.553746 3296 flags.cc:404] Enabled unsafe flag: --never_fsync=true I0107 02:52:34.594192 3124 catalog_manager.cc:1454] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57520: name: "default.loadgen_auto_5e02ede7c9224a8e800a915d3b303249" schema { columns { name: "key" type: INT64 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { } partition_schema { hash_bucket_schemas { columns { name: "key" } num_buckets: 8 seed: 0 } } I0107 02:52:34.614913 3211 tablet_service.cc:1034] Processing CreateTablet for tablet 2c5045da54014b778a4430471a7d67a2 (table=default.loadgen_auto_5e02ede7c9224a8e800a915d3b303249 [id=a7da2c357e484b239049aa1831c588f1]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED I0107 02:52:34.614935 3216 tablet_service.cc:1034] Processing CreateTablet for tablet ce54026443974081b3c3fc942abcb5e5 (table=default.loadgen_auto_5e02ede7c9224a8e800a915d3b303249 [id=a7da2c357e484b239049aa1831c588f1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED I0107 02:52:34.614933 3215 tablet_service.cc:1034] Processing CreateTablet for tablet 11307c61642448418beda42954384b24 (table=default.loadgen_auto_5e02ede7c9224a8e800a915d3b303249 [id=a7da2c357e484b239049aa1831c588f1]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED I0107 02:52:34.615355 3212 tablet_service.cc:1034] Processing CreateTablet for tablet acc0edc926ca4f0fa4a3902edf4eebfc (table=default.loadgen_auto_5e02ede7c9224a8e800a915d3b303249 [id=a7da2c357e484b239049aa1831c588f1]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED I0107 02:52:34.615609 3213 tablet_service.cc:1034] Processing CreateTablet for tablet 5c9836dc106b43fa840744763fc102b4 (table=default.loadgen_auto_5e02ede7c9224a8e800a915d3b303249 [id=a7da2c357e484b239049aa1831c588f1]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED I0107 02:52:34.615823 3214 tablet_service.cc:1034] Processing CreateTablet for tablet 2d4433f026084359aa21efe455d930f8 (table=default.loadgen_auto_5e02ede7c9224a8e800a915d3b303249 [id=a7da2c357e484b239049aa1831c588f1]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED I0107 02:52:34.614913 3218 tablet_service.cc:1034] Processing CreateTablet for tablet 98bbffd7a7e64de8b83b80aa6564a49d (table=default.loadgen_auto_5e02ede7c9224a8e800a915d3b303249 [id=a7da2c357e484b239049aa1831c588f1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED I0107 02:52:34.616259 3217 tablet_service.cc:1034] Processing CreateTablet for tablet e18ce98a75c14fbc84327250c3cfcb47 (table=default.loadgen_auto_5e02ede7c9224a8e800a915d3b303249 [id=a7da2c357e484b239049aa1831c588f1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED I0107 02:52:34.617213 3212 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet acc0edc926ca4f0fa4a3902edf4eebfc. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:34.617518 3213 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 5c9836dc106b43fa840744763fc102b4. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:34.617642 3216 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet ce54026443974081b3c3fc942abcb5e5. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:34.617763 3214 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 2d4433f026084359aa21efe455d930f8. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:34.618547 3211 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 2c5045da54014b778a4430471a7d67a2. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:34.622510 3215 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 11307c61642448418beda42954384b24. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:34.624308 3308 tablet_bootstrap.cc:439] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49: Bootstrap starting. I0107 02:52:34.624873 3217 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet e18ce98a75c14fbc84327250c3cfcb47. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:34.626111 3218 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 98bbffd7a7e64de8b83b80aa6564a49d. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:34.626296 3308 tablet_bootstrap.cc:590] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49: No blocks or log segments found. Creating new log. I0107 02:52:34.627344 3308 log.cc:815] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49: Log is configured to *not* fsync() on all Append() calls I0107 02:52:34.628549 3308 tablet_bootstrap.cc:439] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49: No bootstrap required, opened a new log I0107 02:52:34.628759 3308 ts_tablet_manager.cc:1103] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s I0107 02:52:34.631315 3308 raft_consensus.cc:340] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.631417 3308 raft_consensus.cc:366] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:34.631462 3308 raft_consensus.cc:705] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965d933521a54274adc66bf141c4be49, State: Initialized, Role: FOLLOWER I0107 02:52:34.631884 3308 consensus_queue.cc:231] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.631933 3308 raft_consensus.cc:380] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:34.631983 3308 raft_consensus.cc:472] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:34.632035 3308 raft_consensus.cc:2918] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:34.632722 3308 raft_consensus.cc:494] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.633110 3308 leader_election.cc:307] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 965d933521a54274adc66bf141c4be49; no voters: I0107 02:52:34.633432 3308 leader_election.cc:293] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:34.633947 3290 heartbeater.cc:447] Master 127.2.165.190:40613 was elected leader, sending a full tablet report... I0107 02:52:34.634254 3308 ts_tablet_manager.cc:1131] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s I0107 02:52:34.634586 3310 raft_consensus.cc:2672] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:34.634757 3310 raft_consensus.cc:667] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [term 1 LEADER]: Becoming Leader. State: Replica: 965d933521a54274adc66bf141c4be49, State: Running, Role: LEADER I0107 02:52:34.635095 3310 consensus_queue.cc:208] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.636399 3308 tablet_bootstrap.cc:439] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49: Bootstrap starting. I0107 02:52:34.637370 3308 tablet_bootstrap.cc:590] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49: No blocks or log segments found. Creating new log. I0107 02:52:34.638576 3308 tablet_bootstrap.cc:439] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49: No bootstrap required, opened a new log I0107 02:52:34.638659 3308 ts_tablet_manager.cc:1103] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:34.639274 3308 raft_consensus.cc:340] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.639314 3308 raft_consensus.cc:366] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:34.639330 3308 raft_consensus.cc:705] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965d933521a54274adc66bf141c4be49, State: Initialized, Role: FOLLOWER I0107 02:52:34.639448 3308 consensus_queue.cc:231] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.639497 3308 raft_consensus.cc:380] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:34.639510 3308 raft_consensus.cc:472] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:34.639535 3308 raft_consensus.cc:2918] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:34.639672 3124 catalog_manager.cc:4143] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 reported cstate change: term changed from 0 to 1, leader changed from to 965d933521a54274adc66bf141c4be49 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "965d933521a54274adc66bf141c4be49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } health_report { overall_health: HEALTHY } } } I0107 02:52:34.640473 3308 raft_consensus.cc:494] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.640581 3308 leader_election.cc:307] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 965d933521a54274adc66bf141c4be49; no voters: I0107 02:52:34.640664 3308 leader_election.cc:293] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:34.640754 3312 raft_consensus.cc:2672] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:34.640831 3312 raft_consensus.cc:667] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 [term 1 LEADER]: Becoming Leader. State: Replica: 965d933521a54274adc66bf141c4be49, State: Running, Role: LEADER I0107 02:52:34.640880 3308 ts_tablet_manager.cc:1131] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:34.640971 3312 consensus_queue.cc:208] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.641021 3308 tablet_bootstrap.cc:439] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49: Bootstrap starting. I0107 02:52:34.641976 3308 tablet_bootstrap.cc:590] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49: No blocks or log segments found. Creating new log. I0107 02:52:34.643069 3308 tablet_bootstrap.cc:439] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49: No bootstrap required, opened a new log I0107 02:52:34.643162 3308 ts_tablet_manager.cc:1103] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:34.643581 3308 raft_consensus.cc:340] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.643620 3308 raft_consensus.cc:366] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:34.643635 3308 raft_consensus.cc:705] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965d933521a54274adc66bf141c4be49, State: Initialized, Role: FOLLOWER I0107 02:52:34.643754 3308 consensus_queue.cc:231] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.643831 3308 raft_consensus.cc:380] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:34.643851 3308 raft_consensus.cc:472] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:34.643879 3308 raft_consensus.cc:2918] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:34.644330 3124 catalog_manager.cc:4143] T 5c9836dc106b43fa840744763fc102b4 P 965d933521a54274adc66bf141c4be49 reported cstate change: term changed from 0 to 1, leader changed from to 965d933521a54274adc66bf141c4be49 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "965d933521a54274adc66bf141c4be49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } health_report { overall_health: HEALTHY } } } I0107 02:52:34.644765 3308 raft_consensus.cc:494] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.644874 3308 leader_election.cc:307] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 965d933521a54274adc66bf141c4be49; no voters: I0107 02:52:34.644955 3308 leader_election.cc:293] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:34.645052 3311 raft_consensus.cc:2672] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:34.645166 3311 raft_consensus.cc:667] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 [term 1 LEADER]: Becoming Leader. State: Replica: 965d933521a54274adc66bf141c4be49, State: Running, Role: LEADER I0107 02:52:34.645185 3308 ts_tablet_manager.cc:1131] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:34.645308 3311 consensus_queue.cc:208] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.645360 3308 tablet_bootstrap.cc:439] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49: Bootstrap starting. I0107 02:52:34.646318 3308 tablet_bootstrap.cc:590] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49: No blocks or log segments found. Creating new log. I0107 02:52:34.647505 3308 tablet_bootstrap.cc:439] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49: No bootstrap required, opened a new log I0107 02:52:34.647583 3308 ts_tablet_manager.cc:1103] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:34.648063 3308 raft_consensus.cc:340] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.648099 3308 raft_consensus.cc:366] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:34.648114 3308 raft_consensus.cc:705] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965d933521a54274adc66bf141c4be49, State: Initialized, Role: FOLLOWER I0107 02:52:34.648262 3308 consensus_queue.cc:231] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.648298 3308 raft_consensus.cc:380] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:34.648337 3308 raft_consensus.cc:472] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:34.648367 3308 raft_consensus.cc:2918] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:34.648504 3124 catalog_manager.cc:4143] T acc0edc926ca4f0fa4a3902edf4eebfc P 965d933521a54274adc66bf141c4be49 reported cstate change: term changed from 0 to 1, leader changed from to 965d933521a54274adc66bf141c4be49 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "965d933521a54274adc66bf141c4be49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } health_report { overall_health: HEALTHY } } } I0107 02:52:34.649209 3308 raft_consensus.cc:494] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.649308 3308 leader_election.cc:307] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 965d933521a54274adc66bf141c4be49; no voters: I0107 02:52:34.649385 3308 leader_election.cc:293] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:34.649451 3311 raft_consensus.cc:2672] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:34.649523 3311 raft_consensus.cc:667] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 [term 1 LEADER]: Becoming Leader. State: Replica: 965d933521a54274adc66bf141c4be49, State: Running, Role: LEADER I0107 02:52:34.649616 3308 ts_tablet_manager.cc:1131] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:34.649652 3311 consensus_queue.cc:208] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.649757 3308 tablet_bootstrap.cc:439] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49: Bootstrap starting. I0107 02:52:34.650827 3308 tablet_bootstrap.cc:590] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49: No blocks or log segments found. Creating new log. I0107 02:52:34.651809 3308 tablet_bootstrap.cc:439] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49: No bootstrap required, opened a new log I0107 02:52:34.651895 3308 ts_tablet_manager.cc:1103] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:34.652351 3124 catalog_manager.cc:4143] T 11307c61642448418beda42954384b24 P 965d933521a54274adc66bf141c4be49 reported cstate change: term changed from 0 to 1, leader changed from to 965d933521a54274adc66bf141c4be49 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "965d933521a54274adc66bf141c4be49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } health_report { overall_health: HEALTHY } } } I0107 02:52:34.652452 3308 raft_consensus.cc:340] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.652483 3308 raft_consensus.cc:366] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:34.652498 3308 raft_consensus.cc:705] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965d933521a54274adc66bf141c4be49, State: Initialized, Role: FOLLOWER I0107 02:52:34.652631 3308 consensus_queue.cc:231] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.652665 3308 raft_consensus.cc:380] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:34.652679 3308 raft_consensus.cc:472] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:34.652700 3308 raft_consensus.cc:2918] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:34.653389 3308 raft_consensus.cc:494] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.653517 3308 leader_election.cc:307] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 965d933521a54274adc66bf141c4be49; no voters: I0107 02:52:34.653625 3308 leader_election.cc:293] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:34.653717 3312 raft_consensus.cc:2672] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:34.653841 3308 ts_tablet_manager.cc:1131] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:34.653846 3312 raft_consensus.cc:667] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 [term 1 LEADER]: Becoming Leader. State: Replica: 965d933521a54274adc66bf141c4be49, State: Running, Role: LEADER I0107 02:52:34.653990 3312 consensus_queue.cc:208] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.653992 3308 tablet_bootstrap.cc:439] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49: Bootstrap starting. I0107 02:52:34.655050 3308 tablet_bootstrap.cc:590] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49: No blocks or log segments found. Creating new log. I0107 02:52:34.655864 3124 catalog_manager.cc:4143] T 2c5045da54014b778a4430471a7d67a2 P 965d933521a54274adc66bf141c4be49 reported cstate change: term changed from 0 to 1, leader changed from to 965d933521a54274adc66bf141c4be49 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "965d933521a54274adc66bf141c4be49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } health_report { overall_health: HEALTHY } } } I0107 02:52:34.656653 3308 tablet_bootstrap.cc:439] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49: No bootstrap required, opened a new log I0107 02:52:34.656723 3308 ts_tablet_manager.cc:1103] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s I0107 02:52:34.657155 3308 raft_consensus.cc:340] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.657184 3308 raft_consensus.cc:366] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:34.657196 3308 raft_consensus.cc:705] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965d933521a54274adc66bf141c4be49, State: Initialized, Role: FOLLOWER I0107 02:52:34.657349 3308 consensus_queue.cc:231] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.657380 3308 raft_consensus.cc:380] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:34.657404 3308 raft_consensus.cc:472] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:34.657424 3308 raft_consensus.cc:2918] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:34.658279 3308 raft_consensus.cc:494] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.658392 3308 leader_election.cc:307] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 965d933521a54274adc66bf141c4be49; no voters: I0107 02:52:34.658507 3308 leader_election.cc:293] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:34.658643 3312 raft_consensus.cc:2672] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:34.658733 3308 ts_tablet_manager.cc:1131] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s I0107 02:52:34.658746 3312 raft_consensus.cc:667] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [term 1 LEADER]: Becoming Leader. State: Replica: 965d933521a54274adc66bf141c4be49, State: Running, Role: LEADER I0107 02:52:34.658860 3308 tablet_bootstrap.cc:439] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49: Bootstrap starting. I0107 02:52:34.658881 3312 consensus_queue.cc:208] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.659804 3308 tablet_bootstrap.cc:590] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49: No blocks or log segments found. Creating new log. I0107 02:52:34.660204 3124 catalog_manager.cc:4143] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 reported cstate change: term changed from 0 to 1, leader changed from to 965d933521a54274adc66bf141c4be49 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "965d933521a54274adc66bf141c4be49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } health_report { overall_health: HEALTHY } } } I0107 02:52:34.661037 3308 tablet_bootstrap.cc:439] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49: No bootstrap required, opened a new log I0107 02:52:34.661111 3308 ts_tablet_manager.cc:1103] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s I0107 02:52:34.661574 3308 raft_consensus.cc:340] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.661625 3308 raft_consensus.cc:366] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:34.661643 3308 raft_consensus.cc:705] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965d933521a54274adc66bf141c4be49, State: Initialized, Role: FOLLOWER I0107 02:52:34.661797 3308 consensus_queue.cc:231] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.661823 3308 raft_consensus.cc:380] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:34.661837 3308 raft_consensus.cc:472] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:34.661862 3308 raft_consensus.cc:2918] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:34.662824 3308 raft_consensus.cc:494] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.662931 3308 leader_election.cc:307] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 965d933521a54274adc66bf141c4be49; no voters: I0107 02:52:34.663012 3308 leader_election.cc:293] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:34.663079 3312 raft_consensus.cc:2672] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:34.663130 3312 raft_consensus.cc:667] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 [term 1 LEADER]: Becoming Leader. State: Replica: 965d933521a54274adc66bf141c4be49, State: Running, Role: LEADER I0107 02:52:34.663245 3308 ts_tablet_manager.cc:1131] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s I0107 02:52:34.663328 3312 consensus_queue.cc:208] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.663379 3308 tablet_bootstrap.cc:439] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49: Bootstrap starting. I0107 02:52:34.664767 3308 tablet_bootstrap.cc:590] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49: No blocks or log segments found. Creating new log. I0107 02:52:34.664825 3124 catalog_manager.cc:4143] T 98bbffd7a7e64de8b83b80aa6564a49d P 965d933521a54274adc66bf141c4be49 reported cstate change: term changed from 0 to 1, leader changed from to 965d933521a54274adc66bf141c4be49 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "965d933521a54274adc66bf141c4be49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } health_report { overall_health: HEALTHY } } } I0107 02:52:34.665704 3308 tablet_bootstrap.cc:439] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49: No bootstrap required, opened a new log I0107 02:52:34.665779 3308 ts_tablet_manager.cc:1103] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:34.666182 3308 raft_consensus.cc:340] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.666216 3308 raft_consensus.cc:366] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:34.666231 3308 raft_consensus.cc:705] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965d933521a54274adc66bf141c4be49, State: Initialized, Role: FOLLOWER I0107 02:52:34.666343 3308 consensus_queue.cc:231] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.666378 3308 raft_consensus.cc:380] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:34.666391 3308 raft_consensus.cc:472] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:34.666435 3308 raft_consensus.cc:2918] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:34.667263 3308 raft_consensus.cc:494] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.667376 3308 leader_election.cc:307] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 965d933521a54274adc66bf141c4be49; no voters: I0107 02:52:34.667484 3308 leader_election.cc:293] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:34.667569 3311 raft_consensus.cc:2672] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:34.667639 3311 raft_consensus.cc:667] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 [term 1 LEADER]: Becoming Leader. State: Replica: 965d933521a54274adc66bf141c4be49, State: Running, Role: LEADER I0107 02:52:34.667762 3311 consensus_queue.cc:208] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } } I0107 02:52:34.667878 3308 ts_tablet_manager.cc:1131] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:34.669311 3124 catalog_manager.cc:4143] T 2d4433f026084359aa21efe455d930f8 P 965d933521a54274adc66bf141c4be49 reported cstate change: term changed from 0 to 1, leader changed from to 965d933521a54274adc66bf141c4be49 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "965d933521a54274adc66bf141c4be49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965d933521a54274adc66bf141c4be49" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 37549 } health_report { overall_health: HEALTHY } } } Using auto-created table 'default.loadgen_auto_5e02ede7c9224a8e800a915d3b303249' Generator report time total : 77.3355 ms time per row: 0.0386678 ms Dropping auto-created table 'default.loadgen_auto_5e02ede7c9224a8e800a915d3b303249' I0107 02:52:34.754139 3124 catalog_manager.cc:1950] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57520: table { table_name: "default.loadgen_auto_5e02ede7c9224a8e800a915d3b303249" } modify_external_catalogs: true I0107 02:52:34.757690 3124 catalog_manager.cc:4390] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: Sending DeleteTablet for 1 replicas of tablet 98bbffd7a7e64de8b83b80aa6564a49d I0107 02:52:34.758010 3124 catalog_manager.cc:4390] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: Sending DeleteTablet for 1 replicas of tablet e18ce98a75c14fbc84327250c3cfcb47 I0107 02:52:34.758095 3124 catalog_manager.cc:4390] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: Sending DeleteTablet for 1 replicas of tablet ce54026443974081b3c3fc942abcb5e5 I0107 02:52:34.758154 3124 catalog_manager.cc:4390] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: Sending DeleteTablet for 1 replicas of tablet 11307c61642448418beda42954384b24 I0107 02:52:34.758209 3124 catalog_manager.cc:4390] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: Sending DeleteTablet for 1 replicas of tablet 2d4433f026084359aa21efe455d930f8 I0107 02:52:34.758262 3124 catalog_manager.cc:4390] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: Sending DeleteTablet for 1 replicas of tablet 5c9836dc106b43fa840744763fc102b4 I0107 02:52:34.758312 3124 catalog_manager.cc:4390] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: Sending DeleteTablet for 1 replicas of tablet acc0edc926ca4f0fa4a3902edf4eebfc I0107 02:52:34.758380 3124 catalog_manager.cc:4390] T 00000000000000000000000000000000 P b271f566a92241febcaa7d00d4352f71: Sending DeleteTablet for 1 replicas of tablet 2c5045da54014b778a4430471a7d67a2 I0107 02:52:34.758865 3214 tablet_service.cc:1078] Processing DeleteTablet for tablet e18ce98a75c14fbc84327250c3cfcb47 with delete_type TABLET_DATA_DELETED (Table deleted at 2020-01-07 02:52:34 UTC) from {username='slave'} at 127.0.0.1:58378 I0107 02:52:34.759044 3218 tablet_service.cc:1078] Processing DeleteTablet for tablet ce54026443974081b3c3fc942abcb5e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2020-01-07 02:52:34 UTC) from {username='slave'} at 127.0.0.1:58378 I0107 02:52:34.759157 3212 tablet_service.cc:1078] Processing DeleteTablet for tablet 11307c61642448418beda42954384b24 with delete_type TABLET_DATA_DELETED (Table deleted at 2020-01-07 02:52:34 UTC) from {username='slave'} at 127.0.0.1:58378 I0107 02:52:34.758865 3215 tablet_service.cc:1078] Processing DeleteTablet for tablet 98bbffd7a7e64de8b83b80aa6564a49d with delete_type TABLET_DATA_DELETED (Table deleted at 2020-01-07 02:52:34 UTC) from {username='slave'} at 127.0.0.1:58378 I0107 02:52:34.759227 3216 tablet_service.cc:1078] Processing DeleteTablet for tablet 2d4433f026084359aa21efe455d930f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2020-01-07 02:52:34 UTC) from {username='slave'} at 127.0.0.1:58378 I0107 02:52:34.759284 3217 tablet_service.cc:1078] Processing DeleteTablet for tablet 5c9836dc106b43fa840744763fc102b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2020-01-07 02:52:34 UTC) from {username='slave'} at 127.0.0.1:58378 I0107 02:52:34.759348 3211 tablet_service.cc:1078] Processing DeleteTablet for tablet acc0edc926ca4f0fa4a3902edf4eebfc with delete_type TABLET_DATA_DELETED (Table deleted at 2020-01-07 02:52:34 UTC) from {username='slave'} at 127.0.0.1:58378 I0107 02:52:34.759044 3213 tablet_service.cc:1078] Processing DeleteTablet for tablet 2c5045da54014b778a4430471a7d67a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2020-01-07 02:52:34 UTC) from {username='slave'} at 127.0.0.1:58378 I0107 02:52:34.759464 3343 tablet_replica.cc:282] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49: stopping tablet replica I0107 02:52:34.759716 3343 raft_consensus.cc:2155] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [term 1 LEADER]: Raft consensus shutting down. I0107 02:52:34.759898 3343 raft_consensus.cc:2182] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Raft consensus is shut down! I0107 02:52:34.761318 3343 ts_tablet_manager.cc:1456] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49: Deleting tablet data with delete state TABLET_DATA_DELETED I0107 02:52:34.763965 3343 ts_tablet_manager.cc:1469] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3 I0107 02:52:34.764029 3343 log.cc:1176] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49: Deleting WAL directory at /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenDefaultParameters.1578365549852848-2710/minicluster-data/ts-0/wal/wals/e18ce98a75c14fbc84327250c3cfcb47 I0107 02:52:34.764279 3343 ts_tablet_manager.cc:1490] T e18ce98a75c14fbc84327250c3cfcb47 P 965d933521a54274adc66bf141c4be49: Deleting consensus metadata I0107 02:52:34.765900 3343 tablet_replica.cc:282] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49: stopping tablet replica I0107 02:52:34.766022 3343 raft_consensus.cc:2155] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [term 1 LEADER]: Raft consensus shutting down. I0107 02:52:34.766104 3343 raft_consensus.cc:2182] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49 [term 1 FOLLOWER]: Raft consensus is shut down! I0107 02:52:34.766113 3107 catalog_manager.cc:3512] TS 965d933521a54274adc66bf141c4be49 (127.2.165.129:37549): tablet e18ce98a75c14fbc84327250c3cfcb47 (table default.loadgen_auto_5e02ede7c9224a8e800a915d3b303249 [id=a7da2c357e484b239049aa1831c588f1]) successfully deleted I0107 02:52:34.766777 3343 ts_tablet_manager.cc:1456] T ce54026443974081b3c3fc942abcb5e5 P 965d933521a54274adc66bf141c4be49: Deleting tablet data with delete state TABLET_DATA_DELETED I0107 02:52:34.766777 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 3159 W0107 02:52:34.771123 3107 catalog_manager.cc:3262] TS 965d933521a54274adc66bf141c4be49 (127.2.165.129:37549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c5045da54014b778a4430471a7d67a2: Network error: recv got EOF from 127.2.165.129:37549 (error 108) I0107 02:52:34.771186 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master with pid 3097 2020-01-07T02:52:34Z chronyd exiting [ OK ] ToolTest.TestLoadgenDefaultParameters (645 ms) [ RUN ] ToolTest.TestLoadgenManualFlush 2020-01-07T02:52:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG) 2020-01-07T02:52:34Z Disabled control of system clock I0107 02:52:34.797426 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/data --block_manager=log --webserver_interface=localhost --ipki_ca_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:43071 --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --rpc_reuseport=true --builtin_ntp_servers=127.2.165.148:37121 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:34.864362 3347 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:34.864614 3347 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:34.865675 3347 flags.cc:404] Enabled experimental flag: --ipki_ca_key_size=1024 W0107 02:52:34.865698 3347 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:34.865705 3347 flags.cc:404] Enabled experimental flag: --tsk_num_rsa_bits=512 W0107 02:52:34.865713 3347 flags.cc:404] Enabled experimental flag: --rpc_reuseport=true I0107 02:52:34.867888 3347 master_main.cc:96] Master server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:37121 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/wal --ipki_ca_key_size=1024 --ipki_server_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:43071 --rpc_reuseport=true --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/data/info.pb --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync=true --heap_profile_path=/tmp/kudu-master.3347 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/logs --logbuflevel=-1 --logtostderr=true Master server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:34.877503 3347 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:34.880838 3347 hybrid_clock.cc:186] HybridClock initialized: now 1578365554880786 us; error 33 us; skew 500 ppm I0107 02:52:34.881652 3347 fs_manager.cc:263] Metadata directory not provided I0107 02:52:34.881698 3347 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:34.882004 3347 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:34.882033 3347 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:34.884431 3347 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/data/instance: uuid: "e91529b5de3740e6be6213fa0494f07f" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:34.884753 3347 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/wal/instance: uuid: "e91529b5de3740e6be6213fa0494f07f" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:34.886696 3347 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:34.892802 3347 fs_manager.cc:518] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s I0107 02:52:34.893524 3347 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:34.894047 3347 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:34.898048 3347 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:34.898160 3347 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/wal uuid: "e91529b5de3740e6be6213fa0494f07f" format_stamp: "Formatted at 2020-01-07 02:52:34 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:34.898273 3347 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:34.940943 3347 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:34.948550 3347 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.190:43071 I0107 02:52:34.949690 3347 webserver.cc:382] Webserver started at http://127.2.165.190:43889/ using document root and password file I0107 02:52:34.950397 3347 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/master-0/data/info.pb I0107 02:52:34.952658 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master as pid 3347 I0107 02:52:34.953214 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/data --block_manager=log --rpc_bind_addresses=127.2.165.129:0 --local_ip_for_outbound_sockets=127.2.165.129 --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:43071 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.2.165.148:37121 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} I0107 02:52:34.954460 3408 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:34.961405 3408 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f: Bootstrap starting. I0107 02:52:34.964061 3408 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f: No blocks or log segments found. Creating new log. I0107 02:52:34.965092 3408 log.cc:815] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f: Log is configured to *not* fsync() on all Append() calls I0107 02:52:34.966596 3408 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f: No bootstrap required, opened a new log I0107 02:52:34.969596 3408 raft_consensus.cc:340] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91529b5de3740e6be6213fa0494f07f" member_type: VOTER } I0107 02:52:34.969674 3408 raft_consensus.cc:366] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:34.969713 3408 raft_consensus.cc:705] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91529b5de3740e6be6213fa0494f07f, State: Initialized, Role: FOLLOWER I0107 02:52:34.970265 3408 consensus_queue.cc:231] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91529b5de3740e6be6213fa0494f07f" member_type: VOTER } I0107 02:52:34.970330 3408 raft_consensus.cc:380] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:34.970377 3408 raft_consensus.cc:472] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:34.970494 3408 raft_consensus.cc:2918] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:34.971295 3408 raft_consensus.cc:494] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91529b5de3740e6be6213fa0494f07f" member_type: VOTER } I0107 02:52:34.971758 3408 leader_election.cc:307] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e91529b5de3740e6be6213fa0494f07f; no voters: I0107 02:52:34.972199 3408 leader_election.cc:293] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:34.972352 3411 raft_consensus.cc:2672] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:34.972522 3411 raft_consensus.cc:667] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [term 1 LEADER]: Becoming Leader. State: Replica: e91529b5de3740e6be6213fa0494f07f, State: Running, Role: LEADER I0107 02:52:34.972860 3411 consensus_queue.cc:208] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91529b5de3740e6be6213fa0494f07f" member_type: VOTER } I0107 02:52:34.974532 3411 sys_catalog.cc:339] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e91529b5de3740e6be6213fa0494f07f. Latest consensus state: current_term: 1 leader_uuid: "e91529b5de3740e6be6213fa0494f07f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91529b5de3740e6be6213fa0494f07f" member_type: VOTER } } I0107 02:52:34.974603 3411 sys_catalog.cc:342] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [sys.catalog]: This master's current role is: LEADER I0107 02:52:34.974937 3413 sys_catalog.cc:339] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e91529b5de3740e6be6213fa0494f07f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91529b5de3740e6be6213fa0494f07f" member_type: VOTER } } I0107 02:52:34.974968 3413 sys_catalog.cc:342] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [sys.catalog]: This master's current role is: LEADER I0107 02:52:34.975133 3408 sys_catalog.cc:339] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "e91529b5de3740e6be6213fa0494f07f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91529b5de3740e6be6213fa0494f07f" member_type: VOTER } } I0107 02:52:34.975157 3408 sys_catalog.cc:342] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [sys.catalog]: This master's current role is: LEADER I0107 02:52:34.976379 3408 sys_catalog.cc:436] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f [sys.catalog]: configured and running, proceeding with master startup. W0107 02:52:34.983850 3417 catalog_manager.cc:1158] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f: acquiring CA information for follower catalog manager: Not found: root CA entry not found W0107 02:52:34.983907 3417 catalog_manager.cc:630] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry I0107 02:52:34.983975 3415 catalog_manager.cc:1089] Loading table and tablet metadata into memory... I0107 02:52:34.984992 3415 catalog_manager.cc:1100] Initializing Kudu internal certificate authority... WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:35.014973 3409 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:35.015213 3409 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:35.016177 3409 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:35.016259 3409 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.165.129 I0107 02:52:35.018050 3409 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:37121 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.2.165.129:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:43071 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.3409 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.2.165.129 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:35.027798 3409 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:35.036739 3415 catalog_manager.cc:984] Generated new certificate authority record I0107 02:52:35.037839 3415 catalog_manager.cc:1109] Loading token signing keys... I0107 02:52:35.043375 3409 hybrid_clock.cc:186] HybridClock initialized: now 1578365555043305 us; error 38 us; skew 500 ppm I0107 02:52:35.044363 3409 fs_manager.cc:263] Metadata directory not provided I0107 02:52:35.044430 3409 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:35.044780 3409 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:35.044814 3409 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:35.047870 3409 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/data/instance: uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" format_stamp: "Formatted at 2020-01-07 02:52:35 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:35.048238 3409 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/wal/instance: uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" format_stamp: "Formatted at 2020-01-07 02:52:35 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:35.049774 3415 catalog_manager.cc:4455] T 00000000000000000000000000000000 P e91529b5de3740e6be6213fa0494f07f: Generated new TSK 0 I0107 02:52:35.050362 3415 catalog_manager.cc:1119] Initializing in-progress tserver states... I0107 02:52:35.050807 3409 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:35.059898 3409 fs_manager.cc:518] Time spent creating directory manager: real 0.011s user 0.010s sys 0.000s I0107 02:52:35.060858 3409 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:35.061480 3409 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:35.066371 3409 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:35.066579 3409 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/wal uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" format_stamp: "Formatted at 2020-01-07 02:52:35 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:35.066748 3409 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:35.132309 3409 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:35.134173 3409 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:35.134220 3409 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:35.143343 3409 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.129:42027 I0107 02:52:35.144471 3409 webserver.cc:382] Webserver started at http://127.2.165.129:46761/ using document root and password file I0107 02:52:35.145171 3409 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-0/data/info.pb I0107 02:52:35.154141 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 3409 I0107 02:52:35.154744 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/data --block_manager=log --rpc_bind_addresses=127.2.165.130:0 --local_ip_for_outbound_sockets=127.2.165.130 --webserver_interface=127.2.165.130 --webserver_port=0 --tserver_master_addrs=127.2.165.190:43071 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.2.165.148:37121 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} I0107 02:52:35.158797 3540 heartbeater.cc:322] Connected to a master server at 127.2.165.190:43071 I0107 02:52:35.159014 3540 heartbeater.cc:406] Registering TS with master... I0107 02:52:35.159673 3540 heartbeater.cc:455] Master 127.2.165.190:43071 requested a full tablet report, sending... I0107 02:52:35.161461 3372 ts_manager.cc:192] Registered new tserver with Master: 25c8a86462a347cc8ec0c5ed3ecbb9f8 (127.2.165.129:42027) I0107 02:52:35.163502 3372 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 127.2.165.129:37731 WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:35.212076 3544 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:35.212316 3544 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:35.213259 3544 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:35.213340 3544 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.165.130 I0107 02:52:35.215158 3544 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:37121 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.2.165.130:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/data/info.pb --webserver_interface=127.2.165.130 --webserver_port=0 --tserver_master_addrs=127.2.165.190:43071 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.3544 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.2.165.130 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:35.224483 3544 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:35.225670 3544 hybrid_clock.cc:186] HybridClock initialized: now 1578365555225638 us; error 30 us; skew 500 ppm I0107 02:52:35.226353 3544 fs_manager.cc:263] Metadata directory not provided I0107 02:52:35.226395 3544 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:35.226675 3544 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:35.226698 3544 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:35.228850 3544 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/data/instance: uuid: "4221df31dba44c83a3f7a142a610ffb6" format_stamp: "Formatted at 2020-01-07 02:52:35 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:35.229120 3544 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/wal/instance: uuid: "4221df31dba44c83a3f7a142a610ffb6" format_stamp: "Formatted at 2020-01-07 02:52:35 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:35.230779 3544 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:35.236447 3544 fs_manager.cc:518] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s I0107 02:52:35.237207 3544 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:35.237709 3544 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:35.241472 3544 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:35.241631 3544 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/wal uuid: "4221df31dba44c83a3f7a142a610ffb6" format_stamp: "Formatted at 2020-01-07 02:52:35 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:35.241768 3544 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:35.300073 3544 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:35.301453 3544 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:35.301491 3544 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:35.309463 3544 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.130:43369 I0107 02:52:35.310526 3544 webserver.cc:382] Webserver started at http://127.2.165.130:34303/ using document root and password file I0107 02:52:35.311144 3544 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-1/data/info.pb I0107 02:52:35.311734 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 3544 I0107 02:52:35.312346 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/data --block_manager=log --rpc_bind_addresses=127.2.165.131:0 --local_ip_for_outbound_sockets=127.2.165.131 --webserver_interface=127.2.165.131 --webserver_port=0 --tserver_master_addrs=127.2.165.190:43071 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.2.165.148:37121 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} I0107 02:52:35.323647 3665 heartbeater.cc:322] Connected to a master server at 127.2.165.190:43071 I0107 02:52:35.323863 3665 heartbeater.cc:406] Registering TS with master... I0107 02:52:35.324504 3665 heartbeater.cc:455] Master 127.2.165.190:43071 requested a full tablet report, sending... I0107 02:52:35.325629 3372 ts_manager.cc:192] Registered new tserver with Master: 4221df31dba44c83a3f7a142a610ffb6 (127.2.165.130:43369) I0107 02:52:35.326570 3372 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 127.2.165.130:59685 WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:35.372943 3667 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:35.373184 3667 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:35.374115 3667 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:35.374197 3667 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.165.131 I0107 02:52:35.376065 3667 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:37121 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.2.165.131:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/data/info.pb --webserver_interface=127.2.165.131 --webserver_port=0 --tserver_master_addrs=127.2.165.190:43071 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.3667 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.2.165.131 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:35.385747 3667 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:35.386932 3667 hybrid_clock.cc:186] HybridClock initialized: now 1578365555386855 us; error 73 us; skew 500 ppm I0107 02:52:35.387665 3667 fs_manager.cc:263] Metadata directory not provided I0107 02:52:35.387720 3667 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:35.387998 3667 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:35.388025 3667 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:35.390455 3667 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/data/instance: uuid: "63f09b1a51a6413a920bc0e867463ea8" format_stamp: "Formatted at 2020-01-07 02:52:35 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:35.390796 3667 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/wal/instance: uuid: "63f09b1a51a6413a920bc0e867463ea8" format_stamp: "Formatted at 2020-01-07 02:52:35 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:35.392514 3667 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:35.398298 3667 fs_manager.cc:518] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s I0107 02:52:35.399039 3667 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:35.399518 3667 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:35.403111 3667 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:35.403250 3667 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/wal uuid: "63f09b1a51a6413a920bc0e867463ea8" format_stamp: "Formatted at 2020-01-07 02:52:35 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:35.403401 3667 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:35.445924 3667 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:35.447439 3667 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:35.447471 3667 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:35.455768 3667 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.131:39691 I0107 02:52:35.456735 3667 webserver.cc:382] Webserver started at http://127.2.165.131:33861/ using document root and password file I0107 02:52:35.457348 3667 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenManualFlush.1578365549852848-2710/minicluster-data/ts-2/data/info.pb I0107 02:52:35.460347 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 3667 I0107 02:52:35.466994 3789 heartbeater.cc:322] Connected to a master server at 127.2.165.190:43071 I0107 02:52:35.467178 3789 heartbeater.cc:406] Registering TS with master... I0107 02:52:35.467711 3789 heartbeater.cc:455] Master 127.2.165.190:43071 requested a full tablet report, sending... I0107 02:52:35.468660 3372 ts_manager.cc:192] Registered new tserver with Master: 63f09b1a51a6413a920bc0e867463ea8 (127.2.165.131:39691) I0107 02:52:35.469456 3372 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 127.2.165.131:52283 I0107 02:52:35.470533 2710 external_mini_cluster.cc:721] 3 TS(s) registered with all masters I0107 02:52:35.487046 3372 catalog_manager.cc:1454] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48938: name: "bench_manual_flush" schema { columns { name: "key" type: INT64 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "bool_val" type: BOOL is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int8_val" type: INT8 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int16_val" type: INT16 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int32_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int64_val" type: INT64 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "float_val" type: FLOAT is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "double_val" type: DOUBLE is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "decimal32_val" type: DECIMAL32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 type_attributes { precision: 9 scale: 9 } } columns { name: "decimal64_val" type: DECIMAL64 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 type_attributes { precision: 18 scale: 2 } } columns { name: "decimal128_val" type: DECIMAL128 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 type_attributes { precision: 38 scale: 0 } } columns { name: "unixtime_micros_val" type: UNIXTIME_MICROS is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "binary_val" type: BINARY is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 3 split_rows_range_bounds { } partition_schema { hash_bucket_schemas { columns { name: "key" } num_buckets: 2 seed: 0 } } W0107 02:52:35.488696 3372 catalog_manager.cc:1619] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table bench_manual_flush in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers. I0107 02:52:35.508316 3726 tablet_service.cc:1034] Processing CreateTablet for tablet cc8af90a9a464e149d5476d32964ba8b (table=bench_manual_flush [id=087804814e9f45cbb866359fff15d01f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED I0107 02:52:35.508316 3602 tablet_service.cc:1034] Processing CreateTablet for tablet 758ac5cdcc1042c28e67591fe3e44cd2 (table=bench_manual_flush [id=087804814e9f45cbb866359fff15d01f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED I0107 02:52:35.508316 3477 tablet_service.cc:1034] Processing CreateTablet for tablet cc8af90a9a464e149d5476d32964ba8b (table=bench_manual_flush [id=087804814e9f45cbb866359fff15d01f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED I0107 02:52:35.508316 3475 tablet_service.cc:1034] Processing CreateTablet for tablet 758ac5cdcc1042c28e67591fe3e44cd2 (table=bench_manual_flush [id=087804814e9f45cbb866359fff15d01f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED I0107 02:52:35.509012 3477 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet cc8af90a9a464e149d5476d32964ba8b. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:35.509012 3602 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 758ac5cdcc1042c28e67591fe3e44cd2. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:35.509351 3475 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 758ac5cdcc1042c28e67591fe3e44cd2. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:35.508930 3724 tablet_service.cc:1034] Processing CreateTablet for tablet 758ac5cdcc1042c28e67591fe3e44cd2 (table=bench_manual_flush [id=087804814e9f45cbb866359fff15d01f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED I0107 02:52:35.510191 3726 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet cc8af90a9a464e149d5476d32964ba8b. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:35.510720 3601 tablet_service.cc:1034] Processing CreateTablet for tablet cc8af90a9a464e149d5476d32964ba8b (table=bench_manual_flush [id=087804814e9f45cbb866359fff15d01f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED I0107 02:52:35.510944 3601 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet cc8af90a9a464e149d5476d32964ba8b. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:35.514537 3724 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 758ac5cdcc1042c28e67591fe3e44cd2. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:35.520239 3805 tablet_bootstrap.cc:439] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8: Bootstrap starting. I0107 02:52:35.522349 3805 tablet_bootstrap.cc:590] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8: No blocks or log segments found. Creating new log. I0107 02:52:35.522836 3806 tablet_bootstrap.cc:439] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6: Bootstrap starting. I0107 02:52:35.524665 3806 tablet_bootstrap.cc:590] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6: No blocks or log segments found. Creating new log. I0107 02:52:35.525539 3806 log.cc:815] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6: Log is configured to *not* fsync() on all Append() calls I0107 02:52:35.526378 3807 tablet_bootstrap.cc:439] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8: Bootstrap starting. I0107 02:52:35.528223 3807 tablet_bootstrap.cc:590] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8: No blocks or log segments found. Creating new log. I0107 02:52:35.529037 3807 log.cc:815] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8: Log is configured to *not* fsync() on all Append() calls I0107 02:52:35.530262 3805 log.cc:815] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8: Log is configured to *not* fsync() on all Append() calls I0107 02:52:35.531420 3807 tablet_bootstrap.cc:439] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8: No bootstrap required, opened a new log I0107 02:52:35.531641 3807 ts_tablet_manager.cc:1103] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s I0107 02:52:35.531734 3806 tablet_bootstrap.cc:439] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6: No bootstrap required, opened a new log I0107 02:52:35.531911 3806 ts_tablet_manager.cc:1103] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s I0107 02:52:35.532336 3805 tablet_bootstrap.cc:439] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8: No bootstrap required, opened a new log I0107 02:52:35.532522 3805 ts_tablet_manager.cc:1103] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s I0107 02:52:35.534741 3806 raft_consensus.cc:340] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.534822 3806 raft_consensus.cc:366] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:35.534869 3806 raft_consensus.cc:705] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4221df31dba44c83a3f7a142a610ffb6, State: Initialized, Role: FOLLOWER I0107 02:52:35.534950 3807 raft_consensus.cc:340] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.535037 3807 raft_consensus.cc:366] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:35.535079 3807 raft_consensus.cc:705] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63f09b1a51a6413a920bc0e867463ea8, State: Initialized, Role: FOLLOWER I0107 02:52:35.535389 3806 consensus_queue.cc:231] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.535441 3805 raft_consensus.cc:340] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.535462 3807 consensus_queue.cc:231] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.535524 3805 raft_consensus.cc:366] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:35.535553 3805 raft_consensus.cc:705] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c8a86462a347cc8ec0c5ed3ecbb9f8, State: Initialized, Role: FOLLOWER I0107 02:52:35.535944 3789 heartbeater.cc:447] Master 127.2.165.190:43071 was elected leader, sending a full tablet report... I0107 02:52:35.535953 3805 consensus_queue.cc:231] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.536254 3806 ts_tablet_manager.cc:1131] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s I0107 02:52:35.536391 3540 heartbeater.cc:447] Master 127.2.165.190:43071 was elected leader, sending a full tablet report... I0107 02:52:35.536532 3806 tablet_bootstrap.cc:439] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6: Bootstrap starting. I0107 02:52:35.537158 3807 ts_tablet_manager.cc:1131] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s I0107 02:52:35.537405 3807 tablet_bootstrap.cc:439] T cc8af90a9a464e149d5476d32964ba8b P 63f09b1a51a6413a920bc0e867463ea8: Bootstrap starting. I0107 02:52:35.537796 3806 tablet_bootstrap.cc:590] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6: No blocks or log segments found. Creating new log. I0107 02:52:35.538547 3807 tablet_bootstrap.cc:590] T cc8af90a9a464e149d5476d32964ba8b P 63f09b1a51a6413a920bc0e867463ea8: No blocks or log segments found. Creating new log. I0107 02:52:35.538604 3665 heartbeater.cc:447] Master 127.2.165.190:43071 was elected leader, sending a full tablet report... I0107 02:52:35.539122 3806 tablet_bootstrap.cc:439] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6: No bootstrap required, opened a new log I0107 02:52:35.539237 3806 ts_tablet_manager.cc:1103] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s I0107 02:52:35.540163 3805 ts_tablet_manager.cc:1131] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s I0107 02:52:35.540433 3805 tablet_bootstrap.cc:439] T cc8af90a9a464e149d5476d32964ba8b P 25c8a86462a347cc8ec0c5ed3ecbb9f8: Bootstrap starting. I0107 02:52:35.540623 3807 tablet_bootstrap.cc:439] T cc8af90a9a464e149d5476d32964ba8b P 63f09b1a51a6413a920bc0e867463ea8: No bootstrap required, opened a new log I0107 02:52:35.540740 3807 ts_tablet_manager.cc:1103] T cc8af90a9a464e149d5476d32964ba8b P 63f09b1a51a6413a920bc0e867463ea8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s I0107 02:52:35.541463 3807 raft_consensus.cc:340] T cc8af90a9a464e149d5476d32964ba8b P 63f09b1a51a6413a920bc0e867463ea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.541563 3806 raft_consensus.cc:340] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.541596 3806 raft_consensus.cc:366] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:35.541599 3805 tablet_bootstrap.cc:590] T cc8af90a9a464e149d5476d32964ba8b P 25c8a86462a347cc8ec0c5ed3ecbb9f8: No blocks or log segments found. Creating new log. I0107 02:52:35.541615 3806 raft_consensus.cc:705] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4221df31dba44c83a3f7a142a610ffb6, State: Initialized, Role: FOLLOWER I0107 02:52:35.541779 3806 consensus_queue.cc:231] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.542335 3806 ts_tablet_manager.cc:1131] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s I0107 02:52:35.542769 3807 raft_consensus.cc:366] T cc8af90a9a464e149d5476d32964ba8b P 63f09b1a51a6413a920bc0e867463ea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:35.542812 3807 raft_consensus.cc:705] T cc8af90a9a464e149d5476d32964ba8b P 63f09b1a51a6413a920bc0e867463ea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63f09b1a51a6413a920bc0e867463ea8, State: Initialized, Role: FOLLOWER I0107 02:52:35.543015 3807 consensus_queue.cc:231] T cc8af90a9a464e149d5476d32964ba8b P 63f09b1a51a6413a920bc0e867463ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.543090 3805 tablet_bootstrap.cc:439] T cc8af90a9a464e149d5476d32964ba8b P 25c8a86462a347cc8ec0c5ed3ecbb9f8: No bootstrap required, opened a new log I0107 02:52:35.543197 3805 ts_tablet_manager.cc:1103] T cc8af90a9a464e149d5476d32964ba8b P 25c8a86462a347cc8ec0c5ed3ecbb9f8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s I0107 02:52:35.543367 3807 ts_tablet_manager.cc:1131] T cc8af90a9a464e149d5476d32964ba8b P 63f09b1a51a6413a920bc0e867463ea8: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s I0107 02:52:35.543859 3805 raft_consensus.cc:340] T cc8af90a9a464e149d5476d32964ba8b P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.543897 3805 raft_consensus.cc:366] T cc8af90a9a464e149d5476d32964ba8b P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:35.543915 3805 raft_consensus.cc:705] T cc8af90a9a464e149d5476d32964ba8b P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c8a86462a347cc8ec0c5ed3ecbb9f8, State: Initialized, Role: FOLLOWER I0107 02:52:35.544056 3805 consensus_queue.cc:231] T cc8af90a9a464e149d5476d32964ba8b P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.544598 3805 ts_tablet_manager.cc:1131] T cc8af90a9a464e149d5476d32964ba8b P 25c8a86462a347cc8ec0c5ed3ecbb9f8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s W0107 02:52:35.563009 3666 tablet.cc:1792] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6: Can't schedule compaction. Clean time has not been advanced past its initial value. I0107 02:52:35.618079 3811 raft_consensus.cc:472] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0107 02:52:35.618291 3811 raft_consensus.cc:494] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.619590 3811 leader_election.cc:293] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25c8a86462a347cc8ec0c5ed3ecbb9f8 (127.2.165.129:42027), 63f09b1a51a6413a920bc0e867463ea8 (127.2.165.131:39691) I0107 02:52:35.624303 3812 raft_consensus.cc:472] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0107 02:52:35.624543 3812 raft_consensus.cc:494] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.624858 3746 tablet_service.cc:1307] Received RequestConsensusVote() RPC: tablet_id: "cc8af90a9a464e149d5476d32964ba8b" candidate_uuid: "4221df31dba44c83a3f7a142a610ffb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63f09b1a51a6413a920bc0e867463ea8" is_pre_election: true I0107 02:52:35.625000 3746 raft_consensus.cc:2372] T cc8af90a9a464e149d5476d32964ba8b P 63f09b1a51a6413a920bc0e867463ea8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4221df31dba44c83a3f7a142a610ffb6 in term 0. I0107 02:52:35.625245 3497 tablet_service.cc:1307] Received RequestConsensusVote() RPC: tablet_id: "cc8af90a9a464e149d5476d32964ba8b" candidate_uuid: "4221df31dba44c83a3f7a142a610ffb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" is_pre_election: true I0107 02:52:35.625454 3497 raft_consensus.cc:2372] T cc8af90a9a464e149d5476d32964ba8b P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4221df31dba44c83a3f7a142a610ffb6 in term 0. I0107 02:52:35.625525 3555 leader_election.cc:307] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4221df31dba44c83a3f7a142a610ffb6, 63f09b1a51a6413a920bc0e867463ea8; no voters: I0107 02:52:35.625869 3812 leader_election.cc:293] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25c8a86462a347cc8ec0c5ed3ecbb9f8 (127.2.165.129:42027), 4221df31dba44c83a3f7a142a610ffb6 (127.2.165.130:43369) I0107 02:52:35.625928 3811 raft_consensus.cc:2672] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [term 0 FOLLOWER]: Leader pre-election won for term 1 I0107 02:52:35.625984 3811 raft_consensus.cc:472] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0107 02:52:35.626021 3811 raft_consensus.cc:2918] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:35.627351 3811 raft_consensus.cc:494] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.628082 3497 tablet_service.cc:1307] Received RequestConsensusVote() RPC: tablet_id: "cc8af90a9a464e149d5476d32964ba8b" candidate_uuid: "4221df31dba44c83a3f7a142a610ffb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" I0107 02:52:35.628173 3497 raft_consensus.cc:2918] T cc8af90a9a464e149d5476d32964ba8b P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:35.629321 3497 raft_consensus.cc:2372] T cc8af90a9a464e149d5476d32964ba8b P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4221df31dba44c83a3f7a142a610ffb6 in term 1. I0107 02:52:35.629547 3811 leader_election.cc:293] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [CANDIDATE]: Term 1 election: Requested vote from peers 25c8a86462a347cc8ec0c5ed3ecbb9f8 (127.2.165.129:42027), 63f09b1a51a6413a920bc0e867463ea8 (127.2.165.131:39691) I0107 02:52:35.629683 3555 leader_election.cc:307] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25c8a86462a347cc8ec0c5ed3ecbb9f8, 4221df31dba44c83a3f7a142a610ffb6; no voters: I0107 02:52:35.629842 3811 raft_consensus.cc:2672] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:35.630035 3811 raft_consensus.cc:667] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [term 1 LEADER]: Becoming Leader. State: Replica: 4221df31dba44c83a3f7a142a610ffb6, State: Running, Role: LEADER I0107 02:52:35.630116 3746 tablet_service.cc:1307] Received RequestConsensusVote() RPC: tablet_id: "cc8af90a9a464e149d5476d32964ba8b" candidate_uuid: "4221df31dba44c83a3f7a142a610ffb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63f09b1a51a6413a920bc0e867463ea8" I0107 02:52:35.630183 3746 raft_consensus.cc:2918] T cc8af90a9a464e149d5476d32964ba8b P 63f09b1a51a6413a920bc0e867463ea8 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:35.630275 3497 tablet_service.cc:1307] Received RequestConsensusVote() RPC: tablet_id: "758ac5cdcc1042c28e67591fe3e44cd2" candidate_uuid: "63f09b1a51a6413a920bc0e867463ea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" is_pre_election: true I0107 02:52:35.630348 3497 raft_consensus.cc:2372] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63f09b1a51a6413a920bc0e867463ea8 in term 0. I0107 02:52:35.630461 3811 consensus_queue.cc:208] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.630905 3679 leader_election.cc:307] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25c8a86462a347cc8ec0c5ed3ecbb9f8, 63f09b1a51a6413a920bc0e867463ea8; no voters: I0107 02:52:35.631271 3746 raft_consensus.cc:2372] T cc8af90a9a464e149d5476d32964ba8b P 63f09b1a51a6413a920bc0e867463ea8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4221df31dba44c83a3f7a142a610ffb6 in term 1. I0107 02:52:35.631271 3812 raft_consensus.cc:2672] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [term 0 FOLLOWER]: Leader pre-election won for term 1 I0107 02:52:35.631621 3622 tablet_service.cc:1307] Received RequestConsensusVote() RPC: tablet_id: "758ac5cdcc1042c28e67591fe3e44cd2" candidate_uuid: "63f09b1a51a6413a920bc0e867463ea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4221df31dba44c83a3f7a142a610ffb6" is_pre_election: true I0107 02:52:35.631630 3812 raft_consensus.cc:472] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0107 02:52:35.631656 3812 raft_consensus.cc:2918] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:35.631714 3622 raft_consensus.cc:2372] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63f09b1a51a6413a920bc0e867463ea8 in term 0. I0107 02:52:35.632418 3812 raft_consensus.cc:494] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.632736 3812 leader_election.cc:293] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [CANDIDATE]: Term 1 election: Requested vote from peers 25c8a86462a347cc8ec0c5ed3ecbb9f8 (127.2.165.129:42027), 4221df31dba44c83a3f7a142a610ffb6 (127.2.165.130:43369) I0107 02:52:35.633041 3622 tablet_service.cc:1307] Received RequestConsensusVote() RPC: tablet_id: "758ac5cdcc1042c28e67591fe3e44cd2" candidate_uuid: "63f09b1a51a6413a920bc0e867463ea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4221df31dba44c83a3f7a142a610ffb6" I0107 02:52:35.633103 3622 raft_consensus.cc:2918] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:35.633147 3497 tablet_service.cc:1307] Received RequestConsensusVote() RPC: tablet_id: "758ac5cdcc1042c28e67591fe3e44cd2" candidate_uuid: "63f09b1a51a6413a920bc0e867463ea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" I0107 02:52:35.633210 3497 raft_consensus.cc:2918] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:35.633966 3622 raft_consensus.cc:2372] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63f09b1a51a6413a920bc0e867463ea8 in term 1. I0107 02:52:35.634023 3497 raft_consensus.cc:2372] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63f09b1a51a6413a920bc0e867463ea8 in term 1. I0107 02:52:35.634068 3372 catalog_manager.cc:4143] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 reported cstate change: term changed from 0 to 1, leader changed from to 4221df31dba44c83a3f7a142a610ffb6 (127.2.165.130). New cstate: current_term: 1 leader_uuid: "4221df31dba44c83a3f7a142a610ffb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } health_report { overall_health: UNKNOWN } } } I0107 02:52:35.634266 3677 leader_election.cc:307] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4221df31dba44c83a3f7a142a610ffb6, 63f09b1a51a6413a920bc0e867463ea8; no voters: I0107 02:52:35.634443 3812 raft_consensus.cc:2672] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:35.634629 3812 raft_consensus.cc:667] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [term 1 LEADER]: Becoming Leader. State: Replica: 63f09b1a51a6413a920bc0e867463ea8, State: Running, Role: LEADER I0107 02:52:35.634953 3812 consensus_queue.cc:208] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } } peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } } I0107 02:52:35.637868 3372 catalog_manager.cc:4143] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 reported cstate change: term changed from 0 to 1, leader changed from to 63f09b1a51a6413a920bc0e867463ea8 (127.2.165.131). New cstate: current_term: 1 leader_uuid: "63f09b1a51a6413a920bc0e867463ea8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 } health_report { overall_health: HEALTHY } } } W0107 02:52:35.647013 3541 tablet.cc:1792] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8: Can't schedule compaction. Clean time has not been advanced past its initial value. W0107 02:52:35.708935 3790 tablet.cc:1792] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8: Can't schedule compaction. Clean time has not been advanced past its initial value. W0107 02:52:35.742971 3823 flags.cc:404] Enabled unsafe flag: --never_fsync=true Using table 'bench_manual_flush' I0107 02:52:35.880048 3622 raft_consensus.cc:1184] T 758ac5cdcc1042c28e67591fe3e44cd2 P 4221df31dba44c83a3f7a142a610ffb6 [term 1 FOLLOWER]: Refusing update from remote peer 63f09b1a51a6413a920bc0e867463ea8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch) I0107 02:52:35.880050 3497 raft_consensus.cc:1184] T 758ac5cdcc1042c28e67591fe3e44cd2 P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 1 FOLLOWER]: Refusing update from remote peer 63f09b1a51a6413a920bc0e867463ea8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch) I0107 02:52:35.880929 3812 consensus_queue.cc:984] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s I0107 02:52:35.881448 3821 consensus_queue.cc:984] T 758ac5cdcc1042c28e67591fe3e44cd2 P 63f09b1a51a6413a920bc0e867463ea8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4221df31dba44c83a3f7a142a610ffb6" member_type: VOTER last_known_addr { host: "127.2.165.130" port: 43369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s I0107 02:52:35.883953 3496 raft_consensus.cc:1184] T cc8af90a9a464e149d5476d32964ba8b P 25c8a86462a347cc8ec0c5ed3ecbb9f8 [term 1 FOLLOWER]: Refusing update from remote peer 4221df31dba44c83a3f7a142a610ffb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch) I0107 02:52:35.884539 3819 consensus_queue.cc:984] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25c8a86462a347cc8ec0c5ed3ecbb9f8" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s I0107 02:52:35.887212 3746 raft_consensus.cc:1184] T cc8af90a9a464e149d5476d32964ba8b P 63f09b1a51a6413a920bc0e867463ea8 [term 1 FOLLOWER]: Refusing update from remote peer 4221df31dba44c83a3f7a142a610ffb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch) I0107 02:52:35.887796 3819 consensus_queue.cc:984] T cc8af90a9a464e149d5476d32964ba8b P 4221df31dba44c83a3f7a142a610ffb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63f09b1a51a6413a920bc0e867463ea8" member_type: VOTER last_known_addr { host: "127.2.165.131" port: 39691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s I0107 02:52:35.912982 3842 mvcc.cc:203] Tried to move safe_time back from 6464985316857876480 to 6464985315882532864. Current Snapshot: MvccSnapshot[committed={T|T < 6464985316857876480}] I0107 02:52:35.920140 3839 mvcc.cc:203] Tried to move safe_time back from 6464985316857876480 to 6464985315882532864. Current Snapshot: MvccSnapshot[committed={T|T < 6464985316857876480}] I0107 02:52:35.922741 3845 mvcc.cc:203] Tried to move safe_time back from 6464985316853530624 to 6464985315864944640. Current Snapshot: MvccSnapshot[committed={T|T < 6464985316853530624}] Generator report time total : 682.123 ms time per row: 0.0555113 ms Scanner report expected rows: 12288 actual rows : 12288 I0107 02:52:36.571267 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 3409 I0107 02:52:36.576125 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 3544 I0107 02:52:36.580343 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 3667 I0107 02:52:36.584066 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master with pid 3347 2020-01-07T02:52:36Z chronyd exiting [ OK ] ToolTest.TestLoadgenManualFlush (1814 ms) [ RUN ] ToolTest.TestLoadgenAutoGenTablePartitioning 2020-01-07T02:52:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG) 2020-01-07T02:52:36Z Disabled control of system clock I0107 02:52:36.610635 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/data --block_manager=log --webserver_interface=localhost --ipki_ca_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:35707 --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --rpc_reuseport=true --builtin_ntp_servers=127.2.165.148:39403 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:36.676373 3873 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:36.676589 3873 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:36.677677 3873 flags.cc:404] Enabled experimental flag: --ipki_ca_key_size=1024 W0107 02:52:36.677701 3873 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:36.677708 3873 flags.cc:404] Enabled experimental flag: --tsk_num_rsa_bits=512 W0107 02:52:36.677717 3873 flags.cc:404] Enabled experimental flag: --rpc_reuseport=true I0107 02:52:36.679764 3873 master_main.cc:96] Master server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:39403 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/wal --ipki_ca_key_size=1024 --ipki_server_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:35707 --rpc_reuseport=true --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/data/info.pb --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync=true --heap_profile_path=/tmp/kudu-master.3873 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/logs --logbuflevel=-1 --logtostderr=true Master server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:36.688829 3873 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:36.690006 3873 hybrid_clock.cc:186] HybridClock initialized: now 1578365556689966 us; error 37 us; skew 500 ppm I0107 02:52:36.690726 3873 fs_manager.cc:263] Metadata directory not provided I0107 02:52:36.690768 3873 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:36.691013 3873 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:36.691036 3873 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:36.693205 3873 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/data/instance: uuid: "a065da782f754bb79556ad09ad3ea8f2" format_stamp: "Formatted at 2020-01-07 02:52:36 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:36.693466 3873 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/wal/instance: uuid: "a065da782f754bb79556ad09ad3ea8f2" format_stamp: "Formatted at 2020-01-07 02:52:36 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:36.695165 3873 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:36.700934 3873 fs_manager.cc:518] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s I0107 02:52:36.701624 3873 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:36.702100 3873 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:36.705992 3873 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:36.706091 3873 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/wal uuid: "a065da782f754bb79556ad09ad3ea8f2" format_stamp: "Formatted at 2020-01-07 02:52:36 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:36.706187 3873 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:36.722985 3873 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:36.728457 3873 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.190:35707 I0107 02:52:36.729487 3873 webserver.cc:382] Webserver started at http://127.2.165.190:35867/ using document root and password file I0107 02:52:36.730218 3873 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/master-0/data/info.pb I0107 02:52:36.733916 3934 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:36.735052 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master as pid 3873 I0107 02:52:36.735558 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/data --block_manager=log --rpc_bind_addresses=127.2.165.129:0 --local_ip_for_outbound_sockets=127.2.165.129 --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:35707 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.2.165.148:39403 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} I0107 02:52:36.739878 3934 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2: Bootstrap starting. I0107 02:52:36.742229 3934 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2: No blocks or log segments found. Creating new log. I0107 02:52:36.743207 3934 log.cc:815] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2: Log is configured to *not* fsync() on all Append() calls I0107 02:52:36.744565 3934 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2: No bootstrap required, opened a new log I0107 02:52:36.747529 3934 raft_consensus.cc:340] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a065da782f754bb79556ad09ad3ea8f2" member_type: VOTER } I0107 02:52:36.747606 3934 raft_consensus.cc:366] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:36.747663 3934 raft_consensus.cc:705] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a065da782f754bb79556ad09ad3ea8f2, State: Initialized, Role: FOLLOWER I0107 02:52:36.748175 3934 consensus_queue.cc:231] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a065da782f754bb79556ad09ad3ea8f2" member_type: VOTER } I0107 02:52:36.748236 3934 raft_consensus.cc:380] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:36.748278 3934 raft_consensus.cc:472] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:36.748358 3934 raft_consensus.cc:2918] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:36.749115 3934 raft_consensus.cc:494] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a065da782f754bb79556ad09ad3ea8f2" member_type: VOTER } I0107 02:52:36.749577 3934 leader_election.cc:307] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a065da782f754bb79556ad09ad3ea8f2; no voters: I0107 02:52:36.749999 3934 leader_election.cc:293] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:36.750156 3937 raft_consensus.cc:2672] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:36.750339 3937 raft_consensus.cc:667] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [term 1 LEADER]: Becoming Leader. State: Replica: a065da782f754bb79556ad09ad3ea8f2, State: Running, Role: LEADER I0107 02:52:36.750730 3937 consensus_queue.cc:208] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a065da782f754bb79556ad09ad3ea8f2" member_type: VOTER } I0107 02:52:36.752046 3937 sys_catalog.cc:339] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a065da782f754bb79556ad09ad3ea8f2. Latest consensus state: current_term: 1 leader_uuid: "a065da782f754bb79556ad09ad3ea8f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a065da782f754bb79556ad09ad3ea8f2" member_type: VOTER } } I0107 02:52:36.752087 3937 sys_catalog.cc:342] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [sys.catalog]: This master's current role is: LEADER I0107 02:52:36.752499 3934 sys_catalog.cc:339] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "a065da782f754bb79556ad09ad3ea8f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a065da782f754bb79556ad09ad3ea8f2" member_type: VOTER } } I0107 02:52:36.752530 3934 sys_catalog.cc:342] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [sys.catalog]: This master's current role is: LEADER I0107 02:52:36.753866 3939 sys_catalog.cc:339] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a065da782f754bb79556ad09ad3ea8f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a065da782f754bb79556ad09ad3ea8f2" member_type: VOTER } } I0107 02:52:36.753896 3939 sys_catalog.cc:342] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [sys.catalog]: This master's current role is: LEADER I0107 02:52:36.754319 3934 sys_catalog.cc:436] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2 [sys.catalog]: configured and running, proceeding with master startup. W0107 02:52:36.757385 3943 catalog_manager.cc:1158] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2: acquiring CA information for follower catalog manager: Not found: root CA entry not found W0107 02:52:36.757428 3943 catalog_manager.cc:630] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry I0107 02:52:36.758476 3941 catalog_manager.cc:1089] Loading table and tablet metadata into memory... I0107 02:52:36.759395 3941 catalog_manager.cc:1100] Initializing Kudu internal certificate authority... WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:36.790848 3935 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:36.791069 3935 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:36.792042 3935 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:36.792111 3935 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.165.129 I0107 02:52:36.793850 3935 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:39403 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.2.165.129:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:35707 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.3935 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.2.165.129 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:36.802778 3935 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:36.803969 3935 hybrid_clock.cc:186] HybridClock initialized: now 1578365556803941 us; error 24 us; skew 500 ppm I0107 02:52:36.804563 3935 fs_manager.cc:263] Metadata directory not provided I0107 02:52:36.804603 3935 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:36.804847 3935 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:36.804872 3935 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:36.806993 3935 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/data/instance: uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" format_stamp: "Formatted at 2020-01-07 02:52:36 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:36.807268 3935 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/wal/instance: uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" format_stamp: "Formatted at 2020-01-07 02:52:36 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:36.808852 3935 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:36.814304 3935 fs_manager.cc:518] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s I0107 02:52:36.814967 3935 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:36.815402 3935 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:36.818815 3935 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:36.818949 3935 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/wal uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" format_stamp: "Formatted at 2020-01-07 02:52:36 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:36.819123 3935 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:36.857303 3935 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:36.859059 3935 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:36.859098 3935 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:36.864209 3941 catalog_manager.cc:984] Generated new certificate authority record I0107 02:52:36.865558 3941 catalog_manager.cc:1109] Loading token signing keys... I0107 02:52:36.867780 3935 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.129:33657 I0107 02:52:36.868824 3935 webserver.cc:382] Webserver started at http://127.2.165.129:45489/ using document root and password file I0107 02:52:36.869465 3935 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLoadgenAutoGenTablePartitioning.1578365549852848-2710/minicluster-data/ts-0/data/info.pb I0107 02:52:36.870867 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 3935 I0107 02:52:36.882215 3941 catalog_manager.cc:4455] T 00000000000000000000000000000000 P a065da782f754bb79556ad09ad3ea8f2: Generated new TSK 0 I0107 02:52:36.882836 3941 catalog_manager.cc:1119] Initializing in-progress tserver states... I0107 02:52:36.883477 4066 heartbeater.cc:322] Connected to a master server at 127.2.165.190:35707 I0107 02:52:36.883652 4066 heartbeater.cc:406] Registering TS with master... I0107 02:52:36.884131 4066 heartbeater.cc:455] Master 127.2.165.190:35707 requested a full tablet report, sending... I0107 02:52:36.885532 3901 ts_manager.cc:192] Registered new tserver with Master: 2b6dcad5ff0c481b9a3c7d43499a7454 (127.2.165.129:33657) I0107 02:52:36.887053 3901 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 127.2.165.129:56061 I0107 02:52:36.887501 2710 external_mini_cluster.cc:721] 1 TS(s) registered with all masters W0107 02:52:37.027101 4072 flags.cc:404] Enabled unsafe flag: --never_fsync=true E0107 02:52:37.028986 4072 tool_action_perf.cc:362] Invalid partitioning: --table_num_hash_partitions=1 --table_num_range_partitions=1, must specify more than one partition for auto-generated tables E0107 02:52:37.029043 4072 flags.cc:440] Detected inconsistency in command-line flags; exiting W0107 02:52:37.117604 4076 flags.cc:404] Enabled unsafe flag: --never_fsync=true I0107 02:52:37.155122 3901 catalog_manager.cc:1454] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46912: name: "default.loadgen_auto_2dadbf7e697f48a29c855e1fc6af4805" schema { columns { name: "key" type: INT64 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\224\021\000\000\000\000\000\000" } partition_schema { range_schema { columns { name: "key" } } } I0107 02:52:37.172425 3988 tablet_service.cc:1034] Processing CreateTablet for tablet 30892bfa2cab4a1ea287730febc2afd3 (table=default.loadgen_auto_2dadbf7e697f48a29c855e1fc6af4805 [id=539ee5af74bc406b85a03c5161cb089f]), partition=RANGE (key) PARTITION VALUES >= 4500 I0107 02:52:37.172423 3994 tablet_service.cc:1034] Processing CreateTablet for tablet d54d6c5842b9408a83cc6bf495e8f642 (table=default.loadgen_auto_2dadbf7e697f48a29c855e1fc6af4805 [id=539ee5af74bc406b85a03c5161cb089f]), partition=RANGE (key) PARTITION VALUES < 4500 I0107 02:52:37.173172 3994 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet d54d6c5842b9408a83cc6bf495e8f642. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:37.173637 3988 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 30892bfa2cab4a1ea287730febc2afd3. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:37.179278 4088 tablet_bootstrap.cc:439] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454: Bootstrap starting. I0107 02:52:37.181082 4088 tablet_bootstrap.cc:590] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454: No blocks or log segments found. Creating new log. I0107 02:52:37.181942 4088 log.cc:815] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454: Log is configured to *not* fsync() on all Append() calls I0107 02:52:37.183177 4088 tablet_bootstrap.cc:439] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454: No bootstrap required, opened a new log I0107 02:52:37.183365 4088 ts_tablet_manager.cc:1103] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s I0107 02:52:37.185847 4088 raft_consensus.cc:340] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.185922 4088 raft_consensus.cc:366] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:37.185958 4088 raft_consensus.cc:705] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Initialized, Role: FOLLOWER I0107 02:52:37.186426 4088 consensus_queue.cc:231] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.186486 4088 raft_consensus.cc:380] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:37.186547 4088 raft_consensus.cc:472] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:37.186622 4088 raft_consensus.cc:2918] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:37.187472 4088 raft_consensus.cc:494] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.187921 4088 leader_election.cc:307] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b6dcad5ff0c481b9a3c7d43499a7454; no voters: I0107 02:52:37.188243 4088 leader_election.cc:293] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:37.188410 4090 raft_consensus.cc:2672] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:37.188583 4090 raft_consensus.cc:667] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Running, Role: LEADER I0107 02:52:37.188727 4066 heartbeater.cc:447] Master 127.2.165.190:35707 was elected leader, sending a full tablet report... I0107 02:52:37.188897 4090 consensus_queue.cc:208] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.189029 4088 ts_tablet_manager.cc:1131] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s I0107 02:52:37.189273 4088 tablet_bootstrap.cc:439] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454: Bootstrap starting. I0107 02:52:37.190182 4088 tablet_bootstrap.cc:590] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454: No blocks or log segments found. Creating new log. I0107 02:52:37.191248 4088 tablet_bootstrap.cc:439] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454: No bootstrap required, opened a new log I0107 02:52:37.191332 4088 ts_tablet_manager.cc:1103] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:37.191903 4088 raft_consensus.cc:340] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.191941 4088 raft_consensus.cc:366] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:37.191956 4088 raft_consensus.cc:705] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Initialized, Role: FOLLOWER I0107 02:52:37.192106 4088 consensus_queue.cc:231] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.192143 4088 raft_consensus.cc:380] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:37.192162 4088 raft_consensus.cc:472] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:37.192188 4088 raft_consensus.cc:2918] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:37.192939 4088 raft_consensus.cc:494] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.193017 4088 leader_election.cc:307] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b6dcad5ff0c481b9a3c7d43499a7454; no voters: I0107 02:52:37.193094 4088 leader_election.cc:293] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:37.193114 3901 catalog_manager.cc:4143] T d54d6c5842b9408a83cc6bf495e8f642 P 2b6dcad5ff0c481b9a3c7d43499a7454 reported cstate change: term changed from 0 to 1, leader changed from to 2b6dcad5ff0c481b9a3c7d43499a7454 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } health_report { overall_health: HEALTHY } } } I0107 02:52:37.193146 4092 raft_consensus.cc:2672] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:37.193197 4092 raft_consensus.cc:667] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Running, Role: LEADER I0107 02:52:37.193318 4092 consensus_queue.cc:208] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.193321 4088 ts_tablet_manager.cc:1131] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:37.197515 3901 catalog_manager.cc:4143] T 30892bfa2cab4a1ea287730febc2afd3 P 2b6dcad5ff0c481b9a3c7d43499a7454 reported cstate change: term changed from 0 to 1, leader changed from to 2b6dcad5ff0c481b9a3c7d43499a7454 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } health_report { overall_health: HEALTHY } } } Using auto-created table 'default.loadgen_auto_2dadbf7e697f48a29c855e1fc6af4805' I0107 02:52:37.210182 4099 meta_cache.cc:686] Fast lookup: no cache entry for LookupRpc { table: 'default.loadgen_auto_2dadbf7e697f48a29c855e1fc6af4805', partition-key: (RANGE (key): 4828242564945900421), attempt: 1 }: refreshing our metadata from the Master Generator report time total : 101.482 ms time per row: 0.0338272 ms Scanner report expected rows: 3000 actual rows : 3000 W0107 02:52:37.510445 4110 flags.cc:404] Enabled unsafe flag: --never_fsync=true I0107 02:52:37.544616 3900 catalog_manager.cc:1454] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46920: name: "default.loadgen_auto_edce7d3160484e7e8e19e45d4a07215d" schema { columns { name: "key" type: INT64 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { } partition_schema { hash_bucket_schemas { columns { name: "key" } num_buckets: 2 seed: 0 } } I0107 02:52:37.551491 3988 tablet_service.cc:1034] Processing CreateTablet for tablet dd76326207b54dd79c099d7c05706ea8 (table=default.loadgen_auto_edce7d3160484e7e8e19e45d4a07215d [id=58ec2099ab7643738507860faaf7a3c6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED I0107 02:52:37.551491 3994 tablet_service.cc:1034] Processing CreateTablet for tablet c547a7228c864dcf957b239b0c14cd70 (table=default.loadgen_auto_edce7d3160484e7e8e19e45d4a07215d [id=58ec2099ab7643738507860faaf7a3c6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED I0107 02:52:37.551879 3988 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet dd76326207b54dd79c099d7c05706ea8. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:37.552044 3994 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet c547a7228c864dcf957b239b0c14cd70. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:37.554394 4088 tablet_bootstrap.cc:439] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454: Bootstrap starting. I0107 02:52:37.555454 4088 tablet_bootstrap.cc:590] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454: No blocks or log segments found. Creating new log. I0107 02:52:37.556409 4088 tablet_bootstrap.cc:439] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454: No bootstrap required, opened a new log I0107 02:52:37.556483 4088 ts_tablet_manager.cc:1103] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:37.556912 4088 raft_consensus.cc:340] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.556948 4088 raft_consensus.cc:366] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:37.556967 4088 raft_consensus.cc:705] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Initialized, Role: FOLLOWER I0107 02:52:37.557109 4088 consensus_queue.cc:231] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.557150 4088 raft_consensus.cc:380] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:37.557168 4088 raft_consensus.cc:472] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:37.557190 4088 raft_consensus.cc:2918] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:37.557938 4088 raft_consensus.cc:494] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.558041 4088 leader_election.cc:307] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b6dcad5ff0c481b9a3c7d43499a7454; no voters: I0107 02:52:37.558152 4088 leader_election.cc:293] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:37.558251 4090 raft_consensus.cc:2672] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:37.558328 4090 raft_consensus.cc:667] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Running, Role: LEADER I0107 02:52:37.558398 4088 ts_tablet_manager.cc:1131] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s I0107 02:52:37.558554 4090 consensus_queue.cc:208] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.558568 4088 tablet_bootstrap.cc:439] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454: Bootstrap starting. I0107 02:52:37.559356 4088 tablet_bootstrap.cc:590] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454: No blocks or log segments found. Creating new log. I0107 02:52:37.559882 3900 catalog_manager.cc:4143] T dd76326207b54dd79c099d7c05706ea8 P 2b6dcad5ff0c481b9a3c7d43499a7454 reported cstate change: term changed from 0 to 1, leader changed from to 2b6dcad5ff0c481b9a3c7d43499a7454 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } health_report { overall_health: HEALTHY } } } I0107 02:52:37.560194 4088 tablet_bootstrap.cc:439] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454: No bootstrap required, opened a new log I0107 02:52:37.560283 4088 ts_tablet_manager.cc:1103] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s I0107 02:52:37.560704 4088 raft_consensus.cc:340] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.560740 4088 raft_consensus.cc:366] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:37.560755 4088 raft_consensus.cc:705] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Initialized, Role: FOLLOWER I0107 02:52:37.560911 4088 consensus_queue.cc:231] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.560953 4088 raft_consensus.cc:380] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:37.560967 4088 raft_consensus.cc:472] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:37.560992 4088 raft_consensus.cc:2918] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:37.561758 4088 raft_consensus.cc:494] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.561844 4088 leader_election.cc:307] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b6dcad5ff0c481b9a3c7d43499a7454; no voters: I0107 02:52:37.561931 4088 leader_election.cc:293] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:37.561990 4090 raft_consensus.cc:2672] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:37.562060 4090 raft_consensus.cc:667] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Running, Role: LEADER I0107 02:52:37.562177 4090 consensus_queue.cc:208] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.562204 4088 ts_tablet_manager.cc:1131] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:37.563555 3900 catalog_manager.cc:4143] T c547a7228c864dcf957b239b0c14cd70 P 2b6dcad5ff0c481b9a3c7d43499a7454 reported cstate change: term changed from 0 to 1, leader changed from to 2b6dcad5ff0c481b9a3c7d43499a7454 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } health_report { overall_health: HEALTHY } } } Using auto-created table 'default.loadgen_auto_edce7d3160484e7e8e19e45d4a07215d' Generator report time total : 88.4929 ms time per row: 0.0294976 ms Scanner report expected rows: 3000 actual rows : 3000 W0107 02:52:37.863765 4134 flags.cc:404] Enabled unsafe flag: --never_fsync=true I0107 02:52:37.901549 3899 catalog_manager.cc:1454] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46924: name: "default.loadgen_auto_9c20c2a5d26c495e965151c92a37fe33" schema { columns { name: "key" type: INT64 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { rows: "\004\001\000\224\021\000\000\000\000\000\000" } partition_schema { hash_bucket_schemas { columns { name: "key" } num_buckets: 2 seed: 0 } range_schema { columns { name: "key" } } } I0107 02:52:37.908349 3988 tablet_service.cc:1034] Processing CreateTablet for tablet f561cd2e13b04a06a8b98f9c7e5abc34 (table=default.loadgen_auto_9c20c2a5d26c495e965151c92a37fe33 [id=939accab64184cb98bfefe4e7c506116]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION VALUES < 4500 I0107 02:52:37.908349 3997 tablet_service.cc:1034] Processing CreateTablet for tablet c23da06b48dd43f9b9a059635efd3b11 (table=default.loadgen_auto_9c20c2a5d26c495e965151c92a37fe33 [id=939accab64184cb98bfefe4e7c506116]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION VALUES < 4500 I0107 02:52:37.908349 4003 tablet_service.cc:1034] Processing CreateTablet for tablet 12a0095b60ec4baab9c1c77086c08fb7 (table=default.loadgen_auto_9c20c2a5d26c495e965151c92a37fe33 [id=939accab64184cb98bfefe4e7c506116]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION VALUES >= 4500 I0107 02:52:37.908704 3988 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet f561cd2e13b04a06a8b98f9c7e5abc34. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:37.908929 4003 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 12a0095b60ec4baab9c1c77086c08fb7. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:37.908377 3994 tablet_service.cc:1034] Processing CreateTablet for tablet de6466802d5d4847acd533564e443416 (table=default.loadgen_auto_9c20c2a5d26c495e965151c92a37fe33 [id=939accab64184cb98bfefe4e7c506116]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION VALUES >= 4500 I0107 02:52:37.909152 3994 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet de6466802d5d4847acd533564e443416. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:37.909337 3997 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet c23da06b48dd43f9b9a059635efd3b11. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:37.911093 4088 tablet_bootstrap.cc:439] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454: Bootstrap starting. I0107 02:52:37.912129 4088 tablet_bootstrap.cc:590] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454: No blocks or log segments found. Creating new log. I0107 02:52:37.913656 4088 tablet_bootstrap.cc:439] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454: No bootstrap required, opened a new log I0107 02:52:37.913740 4088 ts_tablet_manager.cc:1103] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s I0107 02:52:37.914216 4088 raft_consensus.cc:340] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.914245 4088 raft_consensus.cc:366] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:37.914259 4088 raft_consensus.cc:705] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Initialized, Role: FOLLOWER I0107 02:52:37.914391 4088 consensus_queue.cc:231] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.914460 4088 raft_consensus.cc:380] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:37.914482 4088 raft_consensus.cc:472] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:37.914502 4088 raft_consensus.cc:2918] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:37.915081 4088 raft_consensus.cc:494] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.915184 4088 leader_election.cc:307] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b6dcad5ff0c481b9a3c7d43499a7454; no voters: I0107 02:52:37.915261 4088 leader_election.cc:293] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:37.915331 4092 raft_consensus.cc:2672] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:37.915467 4092 raft_consensus.cc:667] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Running, Role: LEADER I0107 02:52:37.915592 4092 consensus_queue.cc:208] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.915735 4088 ts_tablet_manager.cc:1131] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s I0107 02:52:37.915865 4088 tablet_bootstrap.cc:439] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454: Bootstrap starting. I0107 02:52:37.916887 4088 tablet_bootstrap.cc:590] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454: No blocks or log segments found. Creating new log. I0107 02:52:37.917117 3899 catalog_manager.cc:4143] T de6466802d5d4847acd533564e443416 P 2b6dcad5ff0c481b9a3c7d43499a7454 reported cstate change: term changed from 0 to 1, leader changed from to 2b6dcad5ff0c481b9a3c7d43499a7454 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } health_report { overall_health: HEALTHY } } } I0107 02:52:37.917942 4088 tablet_bootstrap.cc:439] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454: No bootstrap required, opened a new log I0107 02:52:37.918022 4088 ts_tablet_manager.cc:1103] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s I0107 02:52:37.918438 4088 raft_consensus.cc:340] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.918469 4088 raft_consensus.cc:366] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:37.918488 4088 raft_consensus.cc:705] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Initialized, Role: FOLLOWER I0107 02:52:37.918606 4088 consensus_queue.cc:231] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.918637 4088 raft_consensus.cc:380] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:37.918653 4088 raft_consensus.cc:472] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:37.918679 4088 raft_consensus.cc:2918] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:37.919522 4088 raft_consensus.cc:494] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.919618 4088 leader_election.cc:307] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b6dcad5ff0c481b9a3c7d43499a7454; no voters: I0107 02:52:37.919757 4091 raft_consensus.cc:2672] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:37.919836 4091 raft_consensus.cc:667] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Running, Role: LEADER I0107 02:52:37.919973 4091 consensus_queue.cc:208] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.920413 4088 leader_election.cc:293] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:37.920658 4088 ts_tablet_manager.cc:1131] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s I0107 02:52:37.920800 4088 tablet_bootstrap.cc:439] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454: Bootstrap starting. I0107 02:52:37.921839 4088 tablet_bootstrap.cc:590] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454: No blocks or log segments found. Creating new log. I0107 02:52:37.922205 3899 catalog_manager.cc:4143] T 12a0095b60ec4baab9c1c77086c08fb7 P 2b6dcad5ff0c481b9a3c7d43499a7454 reported cstate change: term changed from 0 to 1, leader changed from to 2b6dcad5ff0c481b9a3c7d43499a7454 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } health_report { overall_health: HEALTHY } } } I0107 02:52:37.923113 4088 tablet_bootstrap.cc:439] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454: No bootstrap required, opened a new log I0107 02:52:37.923187 4088 ts_tablet_manager.cc:1103] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s I0107 02:52:37.923643 4088 raft_consensus.cc:340] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.923676 4088 raft_consensus.cc:366] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:37.923691 4088 raft_consensus.cc:705] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Initialized, Role: FOLLOWER I0107 02:52:37.923825 4088 consensus_queue.cc:231] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.923859 4088 raft_consensus.cc:380] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:37.923874 4088 raft_consensus.cc:472] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:37.923899 4088 raft_consensus.cc:2918] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:37.924787 4088 raft_consensus.cc:494] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.924890 4088 leader_election.cc:307] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b6dcad5ff0c481b9a3c7d43499a7454; no voters: I0107 02:52:37.924969 4088 leader_election.cc:293] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:37.925035 4092 raft_consensus.cc:2672] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:37.925109 4092 raft_consensus.cc:667] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Running, Role: LEADER I0107 02:52:37.925209 4088 ts_tablet_manager.cc:1131] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s I0107 02:52:37.925262 4092 consensus_queue.cc:208] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.925362 4088 tablet_bootstrap.cc:439] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454: Bootstrap starting. I0107 02:52:37.926285 4088 tablet_bootstrap.cc:590] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454: No blocks or log segments found. Creating new log. I0107 02:52:37.926498 3899 catalog_manager.cc:4143] T c23da06b48dd43f9b9a059635efd3b11 P 2b6dcad5ff0c481b9a3c7d43499a7454 reported cstate change: term changed from 0 to 1, leader changed from to 2b6dcad5ff0c481b9a3c7d43499a7454 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } health_report { overall_health: HEALTHY } } } I0107 02:52:37.927254 4088 tablet_bootstrap.cc:439] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454: No bootstrap required, opened a new log I0107 02:52:37.927322 4088 ts_tablet_manager.cc:1103] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s I0107 02:52:37.927678 4088 raft_consensus.cc:340] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.927704 4088 raft_consensus.cc:366] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:37.927716 4088 raft_consensus.cc:705] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Initialized, Role: FOLLOWER I0107 02:52:37.927850 4088 consensus_queue.cc:231] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.927922 4088 raft_consensus.cc:380] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:37.927938 4088 raft_consensus.cc:472] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:37.927968 4088 raft_consensus.cc:2918] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:37.928930 4088 raft_consensus.cc:494] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.929028 4088 leader_election.cc:307] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b6dcad5ff0c481b9a3c7d43499a7454; no voters: I0107 02:52:37.929167 4091 raft_consensus.cc:2672] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:37.929224 4091 raft_consensus.cc:667] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6dcad5ff0c481b9a3c7d43499a7454, State: Running, Role: LEADER I0107 02:52:37.929345 4091 consensus_queue.cc:208] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } } I0107 02:52:37.929760 4088 leader_election.cc:293] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:37.929949 4088 ts_tablet_manager.cc:1131] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s I0107 02:52:37.930985 3899 catalog_manager.cc:4143] T f561cd2e13b04a06a8b98f9c7e5abc34 P 2b6dcad5ff0c481b9a3c7d43499a7454 reported cstate change: term changed from 0 to 1, leader changed from to 2b6dcad5ff0c481b9a3c7d43499a7454 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6dcad5ff0c481b9a3c7d43499a7454" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 33657 } health_report { overall_health: HEALTHY } } } Using auto-created table 'default.loadgen_auto_9c20c2a5d26c495e965151c92a37fe33' Generator report time total : 83.2862 ms time per row: 0.0277621 ms I0107 02:52:38.031327 4157 scanner-internal.cc:555] Opened tablet f561cd2e13b04a06a8b98f9c7e5abc34, no scanner ID assigned Scanner report expected rows: 3000 actual rows : 3000 I0107 02:52:38.135701 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 3935 I0107 02:52:38.140336 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master with pid 3873 2020-01-07T02:52:38Z chronyd exiting [ OK ] ToolTest.TestLoadgenAutoGenTablePartitioning (1555 ms) [ RUN ] ToolTest.TestRemoteReplicaCopy 2020-01-07T02:52:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG) 2020-01-07T02:52:38Z Disabled control of system clock I0107 02:52:38.166333 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/data --block_manager=log --webserver_interface=localhost --ipki_ca_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=0.0.0.0:33341 --webserver_interface=0.0.0.0 --webserver_port=0 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --rpc_reuseport=true --builtin_ntp_servers=127.0.0.1:40087 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin --catalog_manager_wait_for_new_tablets_to_elect_leader=false with env {} WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:38.234527 4163 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:38.234784 4163 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:38.235954 4163 flags.cc:404] Enabled experimental flag: --ipki_ca_key_size=1024 W0107 02:52:38.235981 4163 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:38.235990 4163 flags.cc:404] Enabled experimental flag: --tsk_num_rsa_bits=512 W0107 02:52:38.235998 4163 flags.cc:404] Enabled experimental flag: --rpc_reuseport=true I0107 02:52:38.238000 4163 master_main.cc:96] Master server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.0.0.1:40087 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/wal --catalog_manager_wait_for_new_tablets_to_elect_leader=false --ipki_ca_key_size=1024 --ipki_server_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=0.0.0.0:33341 --rpc_reuseport=true --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/data/info.pb --webserver_interface=0.0.0.0 --webserver_port=0 --never_fsync=true --heap_profile_path=/tmp/kudu-master.4163 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/logs --logbuflevel=-1 --logtostderr=true Master server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:38.247375 4163 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:38.248603 4163 hybrid_clock.cc:186] HybridClock initialized: now 1578365558248561 us; error 29 us; skew 500 ppm I0107 02:52:38.249364 4163 fs_manager.cc:263] Metadata directory not provided I0107 02:52:38.249416 4163 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:38.249729 4163 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:38.249761 4163 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:38.252310 4163 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/data/instance: uuid: "6f23de0579de4c289c99359ae6eb6e6b" format_stamp: "Formatted at 2020-01-07 02:52:38 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:38.252636 4163 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/wal/instance: uuid: "6f23de0579de4c289c99359ae6eb6e6b" format_stamp: "Formatted at 2020-01-07 02:52:38 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:38.254547 4163 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:38.260668 4163 fs_manager.cc:518] Time spent creating directory manager: real 0.008s user 0.005s sys 0.004s I0107 02:52:38.261504 4163 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:38.262056 4163 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:38.266258 4163 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:38.266459 4163 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/wal uuid: "6f23de0579de4c289c99359ae6eb6e6b" format_stamp: "Formatted at 2020-01-07 02:52:38 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:38.266609 4163 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:38.341048 4163 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:38.346992 4163 rpc_server.cc:211] RPC server started. Bound to: 0.0.0.0:33341 I0107 02:52:38.347960 4163 webserver.cc:382] Webserver started at http://0.0.0.0:39009/ using document root and password file I0107 02:52:38.348747 4163 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/master-0/data/info.pb I0107 02:52:38.352407 4224 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:38.352507 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master as pid 4163 I0107 02:52:38.353001 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/data --block_manager=log --rpc_bind_addresses=0.0.0.0:0 --local_ip_for_outbound_sockets=0.0.0.0 --webserver_interface=0.0.0.0 --webserver_port=0 --tserver_master_addrs=dist-test-slave-dist-test-slave-3kpx:33341 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.0.0.1:40087 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin --enable_leader_failure_detection=false with env {} I0107 02:52:38.359701 4224 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b: Bootstrap starting. I0107 02:52:38.363160 4224 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b: No blocks or log segments found. Creating new log. I0107 02:52:38.364445 4224 log.cc:815] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b: Log is configured to *not* fsync() on all Append() calls I0107 02:52:38.366289 4224 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b: No bootstrap required, opened a new log I0107 02:52:38.370365 4224 raft_consensus.cc:340] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f23de0579de4c289c99359ae6eb6e6b" member_type: VOTER } I0107 02:52:38.370491 4224 raft_consensus.cc:366] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:38.370553 4224 raft_consensus.cc:705] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f23de0579de4c289c99359ae6eb6e6b, State: Initialized, Role: FOLLOWER I0107 02:52:38.371206 4224 consensus_queue.cc:231] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f23de0579de4c289c99359ae6eb6e6b" member_type: VOTER } I0107 02:52:38.371282 4224 raft_consensus.cc:380] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:38.371325 4224 raft_consensus.cc:472] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:38.371397 4224 raft_consensus.cc:2918] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:38.372570 4224 raft_consensus.cc:494] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f23de0579de4c289c99359ae6eb6e6b" member_type: VOTER } I0107 02:52:38.373176 4224 leader_election.cc:307] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f23de0579de4c289c99359ae6eb6e6b; no voters: I0107 02:52:38.373708 4224 leader_election.cc:293] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:38.373920 4227 raft_consensus.cc:2672] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:38.374110 4227 raft_consensus.cc:667] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [term 1 LEADER]: Becoming Leader. State: Replica: 6f23de0579de4c289c99359ae6eb6e6b, State: Running, Role: LEADER I0107 02:52:38.374864 4227 consensus_queue.cc:208] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f23de0579de4c289c99359ae6eb6e6b" member_type: VOTER } I0107 02:52:38.376832 4224 sys_catalog.cc:339] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "6f23de0579de4c289c99359ae6eb6e6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f23de0579de4c289c99359ae6eb6e6b" member_type: VOTER } } I0107 02:52:38.376868 4228 sys_catalog.cc:339] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f23de0579de4c289c99359ae6eb6e6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f23de0579de4c289c99359ae6eb6e6b" member_type: VOTER } } I0107 02:52:38.376896 4228 sys_catalog.cc:342] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [sys.catalog]: This master's current role is: LEADER I0107 02:52:38.376952 4229 sys_catalog.cc:339] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f23de0579de4c289c99359ae6eb6e6b. Latest consensus state: current_term: 1 leader_uuid: "6f23de0579de4c289c99359ae6eb6e6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f23de0579de4c289c99359ae6eb6e6b" member_type: VOTER } } I0107 02:52:38.376968 4229 sys_catalog.cc:342] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [sys.catalog]: This master's current role is: LEADER I0107 02:52:38.376874 4224 sys_catalog.cc:342] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [sys.catalog]: This master's current role is: LEADER I0107 02:52:38.378571 4224 sys_catalog.cc:436] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b [sys.catalog]: configured and running, proceeding with master startup. I0107 02:52:38.378715 4231 catalog_manager.cc:1089] Loading table and tablet metadata into memory... I0107 02:52:38.382946 4231 catalog_manager.cc:1100] Initializing Kudu internal certificate authority... I0107 02:52:38.411988 4231 catalog_manager.cc:984] Generated new certificate authority record I0107 02:52:38.413568 4231 catalog_manager.cc:1109] Loading token signing keys... WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:38.415899 4225 flags.cc:404] Enabled unsafe flag: --enable_leader_failure_detection=false W0107 02:52:38.416128 4225 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:38.416170 4225 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:38.417138 4225 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:38.417214 4225 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=0.0.0.0 I0107 02:52:38.419030 4225 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.0.0.1:40087 --ntp_initial_sync_wait_secs=10 --time_source=builtin --enable_leader_failure_detection=false --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/wal --ipki_server_key_size=1024 --rpc_bind_addresses=0.0.0.0:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --webserver_interface=0.0.0.0 --webserver_port=0 --tserver_master_addrs=dist-test-slave-dist-test-slave-3kpx:33341 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.4225 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=0.0.0.0 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:38.426740 4231 catalog_manager.cc:4455] T 00000000000000000000000000000000 P 6f23de0579de4c289c99359ae6eb6e6b: Generated new TSK 0 I0107 02:52:38.427474 4231 catalog_manager.cc:1119] Initializing in-progress tserver states... I0107 02:52:38.429507 4225 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:38.436928 4225 hybrid_clock.cc:186] HybridClock initialized: now 1578365558436865 us; error 37 us; skew 500 ppm I0107 02:52:38.437803 4225 fs_manager.cc:263] Metadata directory not provided I0107 02:52:38.437870 4225 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:38.438202 4225 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:38.438231 4225 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:38.440659 4225 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/data/instance: uuid: "3ac4479875874b51bd79eaf11ddc7556" format_stamp: "Formatted at 2020-01-07 02:52:38 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:38.440979 4225 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/wal/instance: uuid: "3ac4479875874b51bd79eaf11ddc7556" format_stamp: "Formatted at 2020-01-07 02:52:38 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:38.442692 4225 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:38.452008 4225 fs_manager.cc:518] Time spent creating directory manager: real 0.011s user 0.009s sys 0.000s I0107 02:52:38.452849 4225 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:38.453397 4225 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:38.457392 4225 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:38.457535 4225 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/wal uuid: "3ac4479875874b51bd79eaf11ddc7556" format_stamp: "Formatted at 2020-01-07 02:52:38 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:38.457675 4225 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:38.512181 4225 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:38.514006 4225 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:38.514047 4225 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:38.522440 4225 rpc_server.cc:211] RPC server started. Bound to: 0.0.0.0:39053 I0107 02:52:38.523406 4225 webserver.cc:382] Webserver started at http://0.0.0.0:36837/ using document root and password file I0107 02:52:38.524148 4225 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-0/data/info.pb I0107 02:52:38.529440 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 4225 I0107 02:52:38.530004 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data --block_manager=log --rpc_bind_addresses=0.0.0.0:0 --local_ip_for_outbound_sockets=0.0.0.0 --webserver_interface=0.0.0.0 --webserver_port=0 --tserver_master_addrs=dist-test-slave-dist-test-slave-3kpx:33341 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.0.0.1:40087 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin --enable_leader_failure_detection=false with env {} I0107 02:52:38.541359 4356 heartbeater.cc:322] Connected to a master server at dist-test-slave-dist-test-slave-3kpx:33341 I0107 02:52:38.541559 4356 heartbeater.cc:406] Registering TS with master... I0107 02:52:38.542281 4356 heartbeater.cc:455] Master dist-test-slave-dist-test-slave-3kpx:33341 requested a full tablet report, sending... I0107 02:52:38.544229 4183 ts_manager.cc:192] Registered new tserver with Master: 3ac4479875874b51bd79eaf11ddc7556 (dist-test-slave-dist-test-slave-3kpx:39053) I0107 02:52:38.546280 4183 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 172.17.0.2:52155 WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:38.592245 4360 flags.cc:404] Enabled unsafe flag: --enable_leader_failure_detection=false W0107 02:52:38.592470 4360 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:38.592497 4360 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:38.593494 4360 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:38.593559 4360 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=0.0.0.0 I0107 02:52:38.595499 4360 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.0.0.1:40087 --ntp_initial_sync_wait_secs=10 --time_source=builtin --enable_leader_failure_detection=false --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/wal --ipki_server_key_size=1024 --rpc_bind_addresses=0.0.0.0:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data/info.pb --webserver_interface=0.0.0.0 --webserver_port=0 --tserver_master_addrs=dist-test-slave-dist-test-slave-3kpx:33341 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.4360 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=0.0.0.0 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:38.605530 4360 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:38.606753 4360 hybrid_clock.cc:186] HybridClock initialized: now 1578365558606653 us; error 88 us; skew 500 ppm I0107 02:52:38.607447 4360 fs_manager.cc:263] Metadata directory not provided I0107 02:52:38.607497 4360 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:38.607823 4360 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:38.607853 4360 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:38.610054 4360 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data/instance: uuid: "60c3cc7575fb4730ad5eeb12ec456624" format_stamp: "Formatted at 2020-01-07 02:52:38 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:38.610380 4360 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/wal/instance: uuid: "60c3cc7575fb4730ad5eeb12ec456624" format_stamp: "Formatted at 2020-01-07 02:52:38 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:38.612079 4360 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:38.618038 4360 fs_manager.cc:518] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s I0107 02:52:38.618839 4360 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:38.619359 4360 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:38.623210 4360 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:38.623368 4360 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/wal uuid: "60c3cc7575fb4730ad5eeb12ec456624" format_stamp: "Formatted at 2020-01-07 02:52:38 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:38.623509 4360 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:38.678493 4360 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:38.680016 4360 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:38.680052 4360 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:38.687417 4360 rpc_server.cc:211] RPC server started. Bound to: 0.0.0.0:41917 I0107 02:52:38.688336 4360 webserver.cc:382] Webserver started at http://0.0.0.0:36377/ using document root and password file I0107 02:52:38.689014 4360 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data/info.pb I0107 02:52:38.697515 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 4360 I0107 02:52:38.698107 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/data --block_manager=log --rpc_bind_addresses=0.0.0.0:0 --local_ip_for_outbound_sockets=0.0.0.0 --webserver_interface=0.0.0.0 --webserver_port=0 --tserver_master_addrs=dist-test-slave-dist-test-slave-3kpx:33341 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.0.0.1:40087 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin --enable_leader_failure_detection=false with env {} I0107 02:52:38.698779 4481 heartbeater.cc:322] Connected to a master server at dist-test-slave-dist-test-slave-3kpx:33341 I0107 02:52:38.698972 4481 heartbeater.cc:406] Registering TS with master... I0107 02:52:38.699741 4481 heartbeater.cc:455] Master dist-test-slave-dist-test-slave-3kpx:33341 requested a full tablet report, sending... I0107 02:52:38.700855 4183 ts_manager.cc:192] Registered new tserver with Master: 60c3cc7575fb4730ad5eeb12ec456624 (dist-test-slave-dist-test-slave-3kpx:41917) I0107 02:52:38.702024 4183 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 172.17.0.2:59481 WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:38.755076 4484 flags.cc:404] Enabled unsafe flag: --enable_leader_failure_detection=false W0107 02:52:38.755312 4484 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:38.755339 4484 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:38.756239 4484 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:38.756295 4484 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=0.0.0.0 I0107 02:52:38.758078 4484 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.0.0.1:40087 --ntp_initial_sync_wait_secs=10 --time_source=builtin --enable_leader_failure_detection=false --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/wal --ipki_server_key_size=1024 --rpc_bind_addresses=0.0.0.0:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/data/info.pb --webserver_interface=0.0.0.0 --webserver_port=0 --tserver_master_addrs=dist-test-slave-dist-test-slave-3kpx:33341 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.4484 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=0.0.0.0 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:38.768024 4484 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:38.769213 4484 hybrid_clock.cc:186] HybridClock initialized: now 1578365558769170 us; error 36 us; skew 500 ppm I0107 02:52:38.769857 4484 fs_manager.cc:263] Metadata directory not provided I0107 02:52:38.769896 4484 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:38.770157 4484 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:38.770180 4484 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:38.772400 4484 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/data/instance: uuid: "db2c4e3750694192ba003d3a45d4913b" format_stamp: "Formatted at 2020-01-07 02:52:38 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:38.772711 4484 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/wal/instance: uuid: "db2c4e3750694192ba003d3a45d4913b" format_stamp: "Formatted at 2020-01-07 02:52:38 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:38.774350 4484 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:38.780139 4484 fs_manager.cc:518] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s I0107 02:52:38.780897 4484 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:38.781419 4484 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:38.785183 4484 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:38.785370 4484 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/wal uuid: "db2c4e3750694192ba003d3a45d4913b" format_stamp: "Formatted at 2020-01-07 02:52:38 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:38.785509 4484 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:38.819919 4484 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:38.821378 4484 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:38.821411 4484 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:38.830132 4484 rpc_server.cc:211] RPC server started. Bound to: 0.0.0.0:37349 I0107 02:52:38.831246 4484 webserver.cc:382] Webserver started at http://0.0.0.0:42101/ using document root and password file I0107 02:52:38.832026 4484 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-2/data/info.pb I0107 02:52:38.834224 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 4484 I0107 02:52:38.841331 4605 heartbeater.cc:322] Connected to a master server at dist-test-slave-dist-test-slave-3kpx:33341 I0107 02:52:38.841506 4605 heartbeater.cc:406] Registering TS with master... I0107 02:52:38.842170 4605 heartbeater.cc:455] Master dist-test-slave-dist-test-slave-3kpx:33341 requested a full tablet report, sending... I0107 02:52:38.843231 4183 ts_manager.cc:192] Registered new tserver with Master: db2c4e3750694192ba003d3a45d4913b (dist-test-slave-dist-test-slave-3kpx:37349) I0107 02:52:38.844094 4183 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 172.17.0.2:53227 I0107 02:52:38.845175 2710 external_mini_cluster.cc:721] 3 TS(s) registered with all masters I0107 02:52:38.846349 2710 test_util.cc:229] Using random seed: -562695302 I0107 02:52:38.856746 4183 catalog_manager.cc:1454] Servicing CreateTable request from {username='slave'} at 172.17.0.2:57926: name: "test-workload" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 3 split_rows_range_bounds { rows: "\004\001\000\252\252\252*\004\001\000TUUU" } partition_schema { range_schema { columns { name: "key" } } } W0107 02:52:38.858310 4183 catalog_manager.cc:1619] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers. I0107 02:52:38.877069 4290 tablet_service.cc:1034] Processing CreateTablet for tablet 1fafdf1471af4c0cb4862fff1663cdc8 (table=test-workload [id=c1782517762a42b1a745d2481955bf8c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764 I0107 02:52:38.877069 4415 tablet_service.cc:1034] Processing CreateTablet for tablet 5d72ea6e51bb430c823e5230a677deee (table=test-workload [id=c1782517762a42b1a745d2481955bf8c]), partition=RANGE (key) PARTITION VALUES < 715827882 I0107 02:52:38.877069 4529 tablet_service.cc:1034] Processing CreateTablet for tablet 1fafdf1471af4c0cb4862fff1663cdc8 (table=test-workload [id=c1782517762a42b1a745d2481955bf8c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764 I0107 02:52:38.877069 4542 tablet_service.cc:1034] Processing CreateTablet for tablet 528e27054513478bacd3aa83df887395 (table=test-workload [id=c1782517762a42b1a745d2481955bf8c]), partition=RANGE (key) PARTITION VALUES >= 1431655764 I0107 02:52:38.877759 4415 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 5d72ea6e51bb430c823e5230a677deee. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:38.877760 4290 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 1fafdf1471af4c0cb4862fff1663cdc8. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:38.877768 4542 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 528e27054513478bacd3aa83df887395. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:38.878046 4529 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 1fafdf1471af4c0cb4862fff1663cdc8. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:38.878794 4289 tablet_service.cc:1034] Processing CreateTablet for tablet 528e27054513478bacd3aa83df887395 (table=test-workload [id=c1782517762a42b1a745d2481955bf8c]), partition=RANGE (key) PARTITION VALUES >= 1431655764 I0107 02:52:38.878995 4289 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 528e27054513478bacd3aa83df887395. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:38.879022 4530 tablet_service.cc:1034] Processing CreateTablet for tablet 5d72ea6e51bb430c823e5230a677deee (table=test-workload [id=c1782517762a42b1a745d2481955bf8c]), partition=RANGE (key) PARTITION VALUES < 715827882 I0107 02:52:38.879220 4530 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 5d72ea6e51bb430c823e5230a677deee. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:38.879354 4417 tablet_service.cc:1034] Processing CreateTablet for tablet 1fafdf1471af4c0cb4862fff1663cdc8 (table=test-workload [id=c1782517762a42b1a745d2481955bf8c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764 I0107 02:52:38.879545 4417 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 1fafdf1471af4c0cb4862fff1663cdc8. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:38.882792 4418 tablet_service.cc:1034] Processing CreateTablet for tablet 528e27054513478bacd3aa83df887395 (table=test-workload [id=c1782517762a42b1a745d2481955bf8c]), partition=RANGE (key) PARTITION VALUES >= 1431655764 I0107 02:52:38.882839 4278 tablet_service.cc:1034] Processing CreateTablet for tablet 5d72ea6e51bb430c823e5230a677deee (table=test-workload [id=c1782517762a42b1a745d2481955bf8c]), partition=RANGE (key) PARTITION VALUES < 715827882 I0107 02:52:38.882992 4418 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 528e27054513478bacd3aa83df887395. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:38.882992 4278 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 5d72ea6e51bb430c823e5230a677deee. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:38.886204 4621 tablet_bootstrap.cc:439] T 5d72ea6e51bb430c823e5230a677deee P db2c4e3750694192ba003d3a45d4913b: Bootstrap starting. I0107 02:52:38.888299 4621 tablet_bootstrap.cc:590] T 5d72ea6e51bb430c823e5230a677deee P db2c4e3750694192ba003d3a45d4913b: No blocks or log segments found. Creating new log. I0107 02:52:38.889019 4623 tablet_bootstrap.cc:439] T 1fafdf1471af4c0cb4862fff1663cdc8 P 3ac4479875874b51bd79eaf11ddc7556: Bootstrap starting. I0107 02:52:38.890470 4622 tablet_bootstrap.cc:439] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap starting. I0107 02:52:38.891160 4621 log.cc:815] T 5d72ea6e51bb430c823e5230a677deee P db2c4e3750694192ba003d3a45d4913b: Log is configured to *not* fsync() on all Append() calls I0107 02:52:38.893863 4622 tablet_bootstrap.cc:590] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: No blocks or log segments found. Creating new log. I0107 02:52:38.893977 4623 tablet_bootstrap.cc:590] T 1fafdf1471af4c0cb4862fff1663cdc8 P 3ac4479875874b51bd79eaf11ddc7556: No blocks or log segments found. Creating new log. I0107 02:52:38.894748 4621 tablet_bootstrap.cc:439] T 5d72ea6e51bb430c823e5230a677deee P db2c4e3750694192ba003d3a45d4913b: No bootstrap required, opened a new log I0107 02:52:38.894932 4622 log.cc:815] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Log is configured to *not* fsync() on all Append() calls I0107 02:52:38.894970 4621 ts_tablet_manager.cc:1103] T 5d72ea6e51bb430c823e5230a677deee P db2c4e3750694192ba003d3a45d4913b: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s I0107 02:52:38.895007 4623 log.cc:815] T 1fafdf1471af4c0cb4862fff1663cdc8 P 3ac4479875874b51bd79eaf11ddc7556: Log is configured to *not* fsync() on all Append() calls I0107 02:52:38.897660 4621 raft_consensus.cc:340] T 5d72ea6e51bb430c823e5230a677deee P db2c4e3750694192ba003d3a45d4913b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.897753 4621 raft_consensus.cc:705] T 5d72ea6e51bb430c823e5230a677deee P db2c4e3750694192ba003d3a45d4913b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2c4e3750694192ba003d3a45d4913b, State: Initialized, Role: FOLLOWER I0107 02:52:38.898126 4621 consensus_queue.cc:231] T 5d72ea6e51bb430c823e5230a677deee P db2c4e3750694192ba003d3a45d4913b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.899456 4623 tablet_bootstrap.cc:439] T 1fafdf1471af4c0cb4862fff1663cdc8 P 3ac4479875874b51bd79eaf11ddc7556: No bootstrap required, opened a new log I0107 02:52:38.899552 4605 heartbeater.cc:447] Master dist-test-slave-dist-test-slave-3kpx:33341 was elected leader, sending a full tablet report... I0107 02:52:38.899677 4623 ts_tablet_manager.cc:1103] T 1fafdf1471af4c0cb4862fff1663cdc8 P 3ac4479875874b51bd79eaf11ddc7556: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s I0107 02:52:38.899629 4622 tablet_bootstrap.cc:439] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: No bootstrap required, opened a new log I0107 02:52:38.899873 4622 ts_tablet_manager.cc:1103] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s I0107 02:52:38.902634 4623 raft_consensus.cc:340] T 1fafdf1471af4c0cb4862fff1663cdc8 P 3ac4479875874b51bd79eaf11ddc7556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.902737 4623 raft_consensus.cc:705] T 1fafdf1471af4c0cb4862fff1663cdc8 P 3ac4479875874b51bd79eaf11ddc7556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac4479875874b51bd79eaf11ddc7556, State: Initialized, Role: FOLLOWER I0107 02:52:38.902856 4622 raft_consensus.cc:340] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.902940 4622 raft_consensus.cc:705] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c3cc7575fb4730ad5eeb12ec456624, State: Initialized, Role: FOLLOWER I0107 02:52:38.903059 4623 consensus_queue.cc:231] T 1fafdf1471af4c0cb4862fff1663cdc8 P 3ac4479875874b51bd79eaf11ddc7556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.903133 4621 ts_tablet_manager.cc:1131] T 5d72ea6e51bb430c823e5230a677deee P db2c4e3750694192ba003d3a45d4913b: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s I0107 02:52:38.903210 4622 consensus_queue.cc:231] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.903466 4621 tablet_bootstrap.cc:439] T 1fafdf1471af4c0cb4862fff1663cdc8 P db2c4e3750694192ba003d3a45d4913b: Bootstrap starting. I0107 02:52:38.903614 4356 heartbeater.cc:447] Master dist-test-slave-dist-test-slave-3kpx:33341 was elected leader, sending a full tablet report... I0107 02:52:38.903681 4481 heartbeater.cc:447] Master dist-test-slave-dist-test-slave-3kpx:33341 was elected leader, sending a full tablet report... I0107 02:52:38.904724 4621 tablet_bootstrap.cc:590] T 1fafdf1471af4c0cb4862fff1663cdc8 P db2c4e3750694192ba003d3a45d4913b: No blocks or log segments found. Creating new log. I0107 02:52:38.905418 4622 ts_tablet_manager.cc:1131] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s I0107 02:52:38.905418 4623 ts_tablet_manager.cc:1131] T 1fafdf1471af4c0cb4862fff1663cdc8 P 3ac4479875874b51bd79eaf11ddc7556: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s I0107 02:52:38.905700 4623 tablet_bootstrap.cc:439] T 528e27054513478bacd3aa83df887395 P 3ac4479875874b51bd79eaf11ddc7556: Bootstrap starting. I0107 02:52:38.905714 4622 tablet_bootstrap.cc:439] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap starting. I0107 02:52:38.906872 4623 tablet_bootstrap.cc:590] T 528e27054513478bacd3aa83df887395 P 3ac4479875874b51bd79eaf11ddc7556: No blocks or log segments found. Creating new log. I0107 02:52:38.906883 4622 tablet_bootstrap.cc:590] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: No blocks or log segments found. Creating new log. I0107 02:52:38.906883 4621 tablet_bootstrap.cc:439] T 1fafdf1471af4c0cb4862fff1663cdc8 P db2c4e3750694192ba003d3a45d4913b: No bootstrap required, opened a new log I0107 02:52:38.906955 4621 ts_tablet_manager.cc:1103] T 1fafdf1471af4c0cb4862fff1663cdc8 P db2c4e3750694192ba003d3a45d4913b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s I0107 02:52:38.907675 4621 raft_consensus.cc:340] T 1fafdf1471af4c0cb4862fff1663cdc8 P db2c4e3750694192ba003d3a45d4913b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.907717 4621 raft_consensus.cc:705] T 1fafdf1471af4c0cb4862fff1663cdc8 P db2c4e3750694192ba003d3a45d4913b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2c4e3750694192ba003d3a45d4913b, State: Initialized, Role: FOLLOWER I0107 02:52:38.907827 4621 consensus_queue.cc:231] T 1fafdf1471af4c0cb4862fff1663cdc8 P db2c4e3750694192ba003d3a45d4913b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.908718 4622 tablet_bootstrap.cc:439] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: No bootstrap required, opened a new log I0107 02:52:38.908725 4623 tablet_bootstrap.cc:439] T 528e27054513478bacd3aa83df887395 P 3ac4479875874b51bd79eaf11ddc7556: No bootstrap required, opened a new log I0107 02:52:38.908802 4622 ts_tablet_manager.cc:1103] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s I0107 02:52:38.908897 4623 ts_tablet_manager.cc:1103] T 528e27054513478bacd3aa83df887395 P 3ac4479875874b51bd79eaf11ddc7556: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s I0107 02:52:38.909494 4622 raft_consensus.cc:340] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.909533 4622 raft_consensus.cc:705] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c3cc7575fb4730ad5eeb12ec456624, State: Initialized, Role: FOLLOWER I0107 02:52:38.909627 4622 consensus_queue.cc:231] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.909734 4623 raft_consensus.cc:340] T 528e27054513478bacd3aa83df887395 P 3ac4479875874b51bd79eaf11ddc7556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.909814 4623 raft_consensus.cc:705] T 528e27054513478bacd3aa83df887395 P 3ac4479875874b51bd79eaf11ddc7556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac4479875874b51bd79eaf11ddc7556, State: Initialized, Role: FOLLOWER I0107 02:52:38.909956 4623 consensus_queue.cc:231] T 528e27054513478bacd3aa83df887395 P 3ac4479875874b51bd79eaf11ddc7556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.909991 4621 ts_tablet_manager.cc:1131] T 1fafdf1471af4c0cb4862fff1663cdc8 P db2c4e3750694192ba003d3a45d4913b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s I0107 02:52:38.910181 4621 tablet_bootstrap.cc:439] T 528e27054513478bacd3aa83df887395 P db2c4e3750694192ba003d3a45d4913b: Bootstrap starting. I0107 02:52:38.910385 4623 ts_tablet_manager.cc:1131] T 528e27054513478bacd3aa83df887395 P 3ac4479875874b51bd79eaf11ddc7556: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s I0107 02:52:38.910651 4623 tablet_bootstrap.cc:439] T 5d72ea6e51bb430c823e5230a677deee P 3ac4479875874b51bd79eaf11ddc7556: Bootstrap starting. I0107 02:52:38.911268 4621 tablet_bootstrap.cc:590] T 528e27054513478bacd3aa83df887395 P db2c4e3750694192ba003d3a45d4913b: No blocks or log segments found. Creating new log. I0107 02:52:38.911792 4623 tablet_bootstrap.cc:590] T 5d72ea6e51bb430c823e5230a677deee P 3ac4479875874b51bd79eaf11ddc7556: No blocks or log segments found. Creating new log. I0107 02:52:38.912079 4622 ts_tablet_manager.cc:1131] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s I0107 02:52:38.912228 4622 tablet_bootstrap.cc:439] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap starting. I0107 02:52:38.913182 4622 tablet_bootstrap.cc:590] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: No blocks or log segments found. Creating new log. I0107 02:52:38.914186 4623 tablet_bootstrap.cc:439] T 5d72ea6e51bb430c823e5230a677deee P 3ac4479875874b51bd79eaf11ddc7556: No bootstrap required, opened a new log I0107 02:52:38.914260 4623 ts_tablet_manager.cc:1103] T 5d72ea6e51bb430c823e5230a677deee P 3ac4479875874b51bd79eaf11ddc7556: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s I0107 02:52:38.914369 4621 tablet_bootstrap.cc:439] T 528e27054513478bacd3aa83df887395 P db2c4e3750694192ba003d3a45d4913b: No bootstrap required, opened a new log I0107 02:52:38.914480 4621 ts_tablet_manager.cc:1103] T 528e27054513478bacd3aa83df887395 P db2c4e3750694192ba003d3a45d4913b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s I0107 02:52:38.914574 4622 tablet_bootstrap.cc:439] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: No bootstrap required, opened a new log I0107 02:52:38.914638 4622 ts_tablet_manager.cc:1103] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:38.914870 4623 raft_consensus.cc:340] T 5d72ea6e51bb430c823e5230a677deee P 3ac4479875874b51bd79eaf11ddc7556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.914914 4623 raft_consensus.cc:705] T 5d72ea6e51bb430c823e5230a677deee P 3ac4479875874b51bd79eaf11ddc7556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac4479875874b51bd79eaf11ddc7556, State: Initialized, Role: FOLLOWER I0107 02:52:38.915033 4623 consensus_queue.cc:231] T 5d72ea6e51bb430c823e5230a677deee P 3ac4479875874b51bd79eaf11ddc7556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.915088 4622 raft_consensus.cc:340] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.915088 4621 raft_consensus.cc:340] T 528e27054513478bacd3aa83df887395 P db2c4e3750694192ba003d3a45d4913b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.915129 4621 raft_consensus.cc:705] T 528e27054513478bacd3aa83df887395 P db2c4e3750694192ba003d3a45d4913b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2c4e3750694192ba003d3a45d4913b, State: Initialized, Role: FOLLOWER I0107 02:52:38.915151 4622 raft_consensus.cc:705] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c3cc7575fb4730ad5eeb12ec456624, State: Initialized, Role: FOLLOWER I0107 02:52:38.915235 4621 consensus_queue.cc:231] T 528e27054513478bacd3aa83df887395 P db2c4e3750694192ba003d3a45d4913b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.915246 4622 consensus_queue.cc:231] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:38.915338 4623 ts_tablet_manager.cc:1131] T 5d72ea6e51bb430c823e5230a677deee P 3ac4479875874b51bd79eaf11ddc7556: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s I0107 02:52:38.915796 4621 ts_tablet_manager.cc:1131] T 528e27054513478bacd3aa83df887395 P db2c4e3750694192ba003d3a45d4913b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s I0107 02:52:38.915796 4622 ts_tablet_manager.cc:1131] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s W0107 02:52:38.940804 4482 tablet.cc:1792] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: Can't schedule compaction. Clean time has not been advanced past its initial value. W0107 02:52:39.027928 4357 tablet.cc:1792] T 1fafdf1471af4c0cb4862fff1663cdc8 P 3ac4479875874b51bd79eaf11ddc7556: Can't schedule compaction. Clean time has not been advanced past its initial value. W0107 02:52:39.058784 4656 ts_tablet_manager.cc:518] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Tablet Copy: Invalid argument: Leader has replica of tablet 5d72ea6e51bb430c823e5230a677deee with term -1, which is lower than last-logged term 0 on local replica. Rejecting tablet copy request W0107 02:52:39.083523 4606 tablet.cc:1792] T 1fafdf1471af4c0cb4862fff1663cdc8 P db2c4e3750694192ba003d3a45d4913b: Can't schedule compaction. Clean time has not been advanced past its initial value. I0107 02:52:39.189472 4656 tablet_replica.cc:282] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: stopping tablet replica I0107 02:52:39.189810 4656 raft_consensus.cc:2155] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Raft consensus shutting down. I0107 02:52:39.190078 4656 raft_consensus.cc:2182] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Raft consensus is shut down! I0107 02:52:39.191408 4656 ts_tablet_manager.cc:1456] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED I0107 02:52:39.193475 4656 ts_tablet_manager.cc:1469] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 0.0 I0107 02:52:39.193527 4656 log.cc:1176] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Deleting WAL directory at /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/wal/wals/5d72ea6e51bb430c823e5230a677deee I0107 02:52:39.193763 4656 ts_tablet_manager.cc:720] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Initiating tablet copy from peer 3ac4479875874b51bd79eaf11ddc7556 (dist-test-slave-dist-test-slave-3kpx:39053) I0107 02:52:39.194145 4656 tablet_copy_client.cc:243] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: tablet copy: Beginning tablet copy session from remote peer at address dist-test-slave-dist-test-slave-3kpx:39053 I0107 02:52:39.197846 4333 tablet_copy_service.cc:135] P 3ac4479875874b51bd79eaf11ddc7556: Received BeginTabletCopySession request for tablet 5d72ea6e51bb430c823e5230a677deee from peer 60c3cc7575fb4730ad5eeb12ec456624 ({username='slave'} at 172.17.0.2:52625) I0107 02:52:39.197952 4333 tablet_copy_service.cc:156] P 3ac4479875874b51bd79eaf11ddc7556: Beginning new tablet copy session on tablet 5d72ea6e51bb430c823e5230a677deee from peer 60c3cc7575fb4730ad5eeb12ec456624 at {username='slave'} at 172.17.0.2:52625: session id = 60c3cc7575fb4730ad5eeb12ec456624-5d72ea6e51bb430c823e5230a677deee I0107 02:52:39.199313 4333 tablet_copy_source_session.cc:204] T 5d72ea6e51bb430c823e5230a677deee P 3ac4479875874b51bd79eaf11ddc7556: Tablet Copy: opened 0 blocks and 1 log segments I0107 02:52:39.200613 4656 ts_tablet_manager.cc:1456] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Deleting tablet data with delete state TABLET_DATA_COPYING I0107 02:52:39.202226 4656 ts_tablet_manager.cc:1469] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 0.0 I0107 02:52:39.202381 4656 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 5d72ea6e51bb430c823e5230a677deee. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:39.203519 4656 tablet_copy_client.cc:581] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: tablet copy: Starting download of 0 data blocks... I0107 02:52:39.203646 4656 tablet_copy_client.cc:544] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: tablet copy: Starting download of 1 WAL segments... I0107 02:52:39.205029 4656 tablet_copy_client.cc:420] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: tablet copy: Tablet Copy complete. Replacing tablet superblock. I0107 02:52:39.206256 4656 tablet_bootstrap.cc:439] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap starting. I0107 02:52:39.208972 4656 tablet_bootstrap.cc:439] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates I0107 02:52:39.209208 4656 tablet_bootstrap.cc:439] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap complete. I0107 02:52:39.209334 4656 ts_tablet_manager.cc:1103] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Time spent bootstrapping tablet: real 0.003s user 0.004s sys 0.000s I0107 02:52:39.209842 4656 raft_consensus.cc:340] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:39.209879 4656 raft_consensus.cc:705] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c3cc7575fb4730ad5eeb12ec456624, State: Initialized, Role: FOLLOWER I0107 02:52:39.209970 4656 consensus_queue.cc:231] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:39.210211 4656 ts_tablet_manager.cc:1131] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s I0107 02:52:39.210822 4333 tablet_copy_service.cc:325] P 3ac4479875874b51bd79eaf11ddc7556: Request end of tablet copy session 60c3cc7575fb4730ad5eeb12ec456624-5d72ea6e51bb430c823e5230a677deee received from {username='slave'} at 172.17.0.2:52625 I0107 02:52:39.210875 4333 tablet_copy_service.cc:413] P 3ac4479875874b51bd79eaf11ddc7556: ending tablet copy session 60c3cc7575fb4730ad5eeb12ec456624-5d72ea6e51bb430c823e5230a677deee on tablet 5d72ea6e51bb430c823e5230a677deee with peer 60c3cc7575fb4730ad5eeb12ec456624 I0107 02:52:39.222360 4415 tablet_service.cc:1078] Processing DeleteTablet for tablet 528e27054513478bacd3aa83df887395 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 172.17.0.2:49098 I0107 02:52:39.222822 4678 tablet_replica.cc:282] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: stopping tablet replica I0107 02:52:39.222946 4678 raft_consensus.cc:2155] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Raft consensus shutting down. I0107 02:52:39.223047 4678 raft_consensus.cc:2182] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Raft consensus is shut down! I0107 02:52:39.223698 4678 ts_tablet_manager.cc:1456] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED I0107 02:52:39.225651 4678 ts_tablet_manager.cc:1469] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 0.0 I0107 02:52:39.225716 4678 log.cc:1176] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Deleting WAL directory at /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/wal/wals/528e27054513478bacd3aa83df887395 I0107 02:52:39.226460 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 4360 I0107 02:52:39.353233 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data --block_manager=log --rpc_bind_addresses=0.0.0.0:41917 --local_ip_for_outbound_sockets=0.0.0.0 --webserver_port=36377 --webserver_interface=0.0.0.0 --tserver_master_addrs=dist-test-slave-dist-test-slave-3kpx:33341 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.0.0.1:40087 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin --enable_leader_failure_detection=false with env {} WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:39.411661 4686 flags.cc:404] Enabled unsafe flag: --enable_leader_failure_detection=false W0107 02:52:39.411898 4686 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:39.411931 4686 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:39.413072 4686 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:39.413151 4686 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=0.0.0.0 I0107 02:52:39.415159 4686 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.0.0.1:40087 --ntp_initial_sync_wait_secs=10 --time_source=builtin --enable_leader_failure_detection=false --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/wal --ipki_server_key_size=1024 --rpc_bind_addresses=0.0.0.0:41917 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data/info.pb --webserver_interface=0.0.0.0 --webserver_port=36377 --tserver_master_addrs=dist-test-slave-dist-test-slave-3kpx:33341 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.4686 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=0.0.0.0 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:39.425741 4686 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:39.426965 4686 hybrid_clock.cc:186] HybridClock initialized: now 1578365559426920 us; error 33 us; skew 500 ppm I0107 02:52:39.427665 4686 fs_manager.cc:263] Metadata directory not provided I0107 02:52:39.427716 4686 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:39.434331 4686 fs_manager.cc:414] Time spent opening directory manager: real 0.005s user 0.002s sys 0.005s I0107 02:52:39.434906 4686 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:39.435410 4686 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:39.439179 4686 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:39.439934 4686 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/wal uuid: "60c3cc7575fb4730ad5eeb12ec456624" format_stamp: "Formatted at 2020-01-07 02:52:38 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:39.440090 4686 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:39.491858 4686 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:39.493423 4686 ts_tablet_manager.cc:383] Loading tablet metadata (0/2 complete) I0107 02:52:39.498808 4686 ts_tablet_manager.cc:395] Loaded tablet metadata (2 total tablets, 1 live tablets) I0107 02:52:39.498847 4686 ts_tablet_manager.cc:401] Registering tablets (0/1 complete) I0107 02:52:39.499585 4686 ts_tablet_manager.cc:415] Registered 1 tablets I0107 02:52:39.500041 4703 tablet_bootstrap.cc:439] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap starting. I0107 02:52:39.508116 4686 rpc_server.cc:211] RPC server started. Bound to: 0.0.0.0:41917 I0107 02:52:39.509112 4686 webserver.cc:382] Webserver started at http://0.0.0.0:36377/ using document root and password file I0107 02:52:39.509922 4686 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRemoteReplicaCopy.1578365549852848-2710/minicluster-data/ts-1/data/info.pb I0107 02:52:39.516830 4703 log.cc:815] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Log is configured to *not* fsync() on all Append() calls I0107 02:52:39.518532 4703 tablet_bootstrap.cc:439] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates I0107 02:52:39.518929 4703 tablet_bootstrap.cc:439] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap complete. I0107 02:52:39.519037 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 4686 I0107 02:52:39.520066 4703 ts_tablet_manager.cc:1103] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.006s I0107 02:52:39.521167 4808 heartbeater.cc:322] Connected to a master server at dist-test-slave-dist-test-slave-3kpx:33341 I0107 02:52:39.521396 4808 heartbeater.cc:406] Registering TS with master... I0107 02:52:39.522218 4808 heartbeater.cc:455] Master dist-test-slave-dist-test-slave-3kpx:33341 requested a full tablet report, sending... I0107 02:52:39.524222 4703 raft_consensus.cc:340] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:39.524245 4183 ts_manager.cc:192] Re-registered known tserver with Master: 60c3cc7575fb4730ad5eeb12ec456624 (dist-test-slave-dist-test-slave-3kpx:41917) I0107 02:52:39.524323 4703 raft_consensus.cc:705] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c3cc7575fb4730ad5eeb12ec456624, State: Initialized, Role: FOLLOWER I0107 02:52:39.524631 4703 consensus_queue.cc:231] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:39.525336 4703 ts_tablet_manager.cc:1131] T 5d72ea6e51bb430c823e5230a677deee P 60c3cc7575fb4730ad5eeb12ec456624: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s I0107 02:52:39.525630 4183 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 172.17.0.2:52765 I0107 02:52:39.527262 4808 heartbeater.cc:447] Master dist-test-slave-dist-test-slave-3kpx:33341 was elected leader, sending a full tablet report... I0107 02:52:39.652348 4836 tablet_replica.cc:282] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: stopping tablet replica I0107 02:52:39.652483 4836 raft_consensus.cc:2155] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Raft consensus shutting down. I0107 02:52:39.652578 4836 raft_consensus.cc:2182] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Raft consensus is shut down! I0107 02:52:39.652719 4836 ts_tablet_manager.cc:720] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Initiating tablet copy from peer 3ac4479875874b51bd79eaf11ddc7556 (dist-test-slave-dist-test-slave-3kpx:39053) I0107 02:52:39.652918 4836 tablet_copy_client.cc:206] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: tablet copy: overwriting existing tombstoned replica with an unknown last-logged opid I0107 02:52:39.653115 4836 tablet_copy_client.cc:243] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: tablet copy: Beginning tablet copy session from remote peer at address dist-test-slave-dist-test-slave-3kpx:39053 I0107 02:52:39.656713 4333 tablet_copy_service.cc:135] P 3ac4479875874b51bd79eaf11ddc7556: Received BeginTabletCopySession request for tablet 528e27054513478bacd3aa83df887395 from peer 60c3cc7575fb4730ad5eeb12ec456624 ({username='slave'} at 172.17.0.2:44179) I0107 02:52:39.656795 4333 tablet_copy_service.cc:156] P 3ac4479875874b51bd79eaf11ddc7556: Beginning new tablet copy session on tablet 528e27054513478bacd3aa83df887395 from peer 60c3cc7575fb4730ad5eeb12ec456624 at {username='slave'} at 172.17.0.2:44179: session id = 60c3cc7575fb4730ad5eeb12ec456624-528e27054513478bacd3aa83df887395 I0107 02:52:39.657618 4333 tablet_copy_source_session.cc:204] T 528e27054513478bacd3aa83df887395 P 3ac4479875874b51bd79eaf11ddc7556: Tablet Copy: opened 0 blocks and 1 log segments I0107 02:52:39.658926 4836 ts_tablet_manager.cc:1456] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Deleting tablet data with delete state TABLET_DATA_COPYING I0107 02:52:39.661687 4836 ts_tablet_manager.cc:1469] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId unknown I0107 02:52:39.662001 4836 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 528e27054513478bacd3aa83df887395. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:39.663321 4836 tablet_copy_client.cc:581] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: tablet copy: Starting download of 0 data blocks... I0107 02:52:39.663430 4836 tablet_copy_client.cc:544] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: tablet copy: Starting download of 1 WAL segments... I0107 02:52:39.664778 4836 tablet_copy_client.cc:420] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: tablet copy: Tablet Copy complete. Replacing tablet superblock. I0107 02:52:39.666252 4836 tablet_bootstrap.cc:439] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap starting. I0107 02:52:39.669739 4836 tablet_bootstrap.cc:439] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates I0107 02:52:39.669936 4836 tablet_bootstrap.cc:439] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap complete. I0107 02:52:39.670074 4836 ts_tablet_manager.cc:1103] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s I0107 02:52:39.670773 4836 raft_consensus.cc:340] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:39.670809 4836 raft_consensus.cc:705] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c3cc7575fb4730ad5eeb12ec456624, State: Initialized, Role: FOLLOWER I0107 02:52:39.670900 4836 consensus_queue.cc:231] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:39.671133 4836 ts_tablet_manager.cc:1131] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s I0107 02:52:39.671855 4333 tablet_copy_service.cc:325] P 3ac4479875874b51bd79eaf11ddc7556: Request end of tablet copy session 60c3cc7575fb4730ad5eeb12ec456624-528e27054513478bacd3aa83df887395 received from {username='slave'} at 172.17.0.2:44179 I0107 02:52:39.671903 4333 tablet_copy_service.cc:413] P 3ac4479875874b51bd79eaf11ddc7556: ending tablet copy session 60c3cc7575fb4730ad5eeb12ec456624-528e27054513478bacd3aa83df887395 on tablet 528e27054513478bacd3aa83df887395 with peer 60c3cc7575fb4730ad5eeb12ec456624 W0107 02:52:39.760921 4809 tablet.cc:1792] T 528e27054513478bacd3aa83df887395 P 60c3cc7575fb4730ad5eeb12ec456624: Can't schedule compaction. Clean time has not been advanced past its initial value. I0107 02:52:39.806464 4836 ts_tablet_manager.cc:720] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: Initiating tablet copy from peer 3ac4479875874b51bd79eaf11ddc7556 (dist-test-slave-dist-test-slave-3kpx:39053) I0107 02:52:39.806720 4836 tablet_copy_client.cc:243] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: tablet copy: Beginning tablet copy session from remote peer at address dist-test-slave-dist-test-slave-3kpx:39053 I0107 02:52:39.807335 4333 tablet_copy_service.cc:135] P 3ac4479875874b51bd79eaf11ddc7556: Received BeginTabletCopySession request for tablet 1fafdf1471af4c0cb4862fff1663cdc8 from peer 60c3cc7575fb4730ad5eeb12ec456624 ({username='slave'} at 172.17.0.2:44179) I0107 02:52:39.807441 4333 tablet_copy_service.cc:156] P 3ac4479875874b51bd79eaf11ddc7556: Beginning new tablet copy session on tablet 1fafdf1471af4c0cb4862fff1663cdc8 from peer 60c3cc7575fb4730ad5eeb12ec456624 at {username='slave'} at 172.17.0.2:44179: session id = 60c3cc7575fb4730ad5eeb12ec456624-1fafdf1471af4c0cb4862fff1663cdc8 I0107 02:52:39.808218 4333 tablet_copy_source_session.cc:204] T 1fafdf1471af4c0cb4862fff1663cdc8 P 3ac4479875874b51bd79eaf11ddc7556: Tablet Copy: opened 0 blocks and 1 log segments I0107 02:52:39.809199 4836 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 1fafdf1471af4c0cb4862fff1663cdc8. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:39.811307 4836 tablet_copy_client.cc:581] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: tablet copy: Starting download of 0 data blocks... I0107 02:52:39.811408 4836 tablet_copy_client.cc:544] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: tablet copy: Starting download of 1 WAL segments... I0107 02:52:39.812460 4836 tablet_copy_client.cc:420] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: tablet copy: Tablet Copy complete. Replacing tablet superblock. I0107 02:52:39.813652 4836 tablet_bootstrap.cc:439] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap starting. I0107 02:52:39.815979 4836 tablet_bootstrap.cc:439] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates I0107 02:52:39.816162 4836 tablet_bootstrap.cc:439] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: Bootstrap complete. I0107 02:52:39.816293 4836 ts_tablet_manager.cc:1103] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s I0107 02:52:39.816800 4836 raft_consensus.cc:340] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:39.816841 4836 raft_consensus.cc:705] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c3cc7575fb4730ad5eeb12ec456624, State: Initialized, Role: FOLLOWER I0107 02:52:39.816942 4836 consensus_queue.cc:231] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c3cc7575fb4730ad5eeb12ec456624" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 41917 } } peers { permanent_uuid: "3ac4479875874b51bd79eaf11ddc7556" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 39053 } } peers { permanent_uuid: "db2c4e3750694192ba003d3a45d4913b" member_type: VOTER last_known_addr { host: "dist-test-slave-dist-test-slave-3kpx" port: 37349 } } I0107 02:52:39.817178 4836 ts_tablet_manager.cc:1131] T 1fafdf1471af4c0cb4862fff1663cdc8 P 60c3cc7575fb4730ad5eeb12ec456624: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s I0107 02:52:39.817724 4333 tablet_copy_service.cc:325] P 3ac4479875874b51bd79eaf11ddc7556: Request end of tablet copy session 60c3cc7575fb4730ad5eeb12ec456624-1fafdf1471af4c0cb4862fff1663cdc8 received from {username='slave'} at 172.17.0.2:44179 I0107 02:52:39.817768 4333 tablet_copy_service.cc:413] P 3ac4479875874b51bd79eaf11ddc7556: ending tablet copy session 60c3cc7575fb4730ad5eeb12ec456624-1fafdf1471af4c0cb4862fff1663cdc8 on tablet 1fafdf1471af4c0cb4862fff1663cdc8 with peer 60c3cc7575fb4730ad5eeb12ec456624 I0107 02:52:39.818979 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 4225 I0107 02:52:39.823719 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 4686 I0107 02:52:39.827780 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 4484 I0107 02:52:39.831352 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master with pid 4163 2020-01-07T02:52:39Z chronyd exiting [ OK ] ToolTest.TestRemoteReplicaCopy (1692 ms) [ RUN ] ToolTest.TestLocalReplicaCMetaOps I0107 02:52:39.850344 2710 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.2.165.190:43261 W0107 02:52:39.857347 2710 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters. I0107 02:52:39.857484 2710 hybrid_clock.cc:186] HybridClock initialized: now 1578365559857462 us; error 0 us; skew 500 ppm I0107 02:52:39.857653 2710 fs_manager.cc:263] Metadata directory not provided I0107 02:52:39.857689 2710 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:39.857787 2710 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:39.857836 2710 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:39.858314 2710 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/master-0-root/instance: uuid: "151db0fe99064b19ba0d87577bb11ff6" format_stamp: "Formatted at 2020-01-07 02:52:39 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:39.859630 2710 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/master-0-root/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/master-0-root/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:39.862991 2710 fs_manager.cc:518] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s I0107 02:52:39.863806 2710 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:39.865787 2710 fs_manager.cc:434] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s I0107 02:52:39.865856 2710 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/master-0-root uuid: "151db0fe99064b19ba0d87577bb11ff6" format_stamp: "Formatted at 2020-01-07 02:52:39 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:39.865911 2710 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:39.880965 2710 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:39.887867 2710 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.190:43261 I0107 02:52:39.889088 2710 webserver.cc:382] Webserver started at http://127.2.165.190:38871/ using document root and password file I0107 02:52:39.890612 4914 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:39.894316 4914 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6: Bootstrap starting. I0107 02:52:39.896812 4914 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6: No blocks or log segments found. Creating new log. I0107 02:52:39.897945 4914 log.cc:815] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6: Log is configured to *not* fsync() on all Append() calls I0107 02:52:39.899566 4914 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6: No bootstrap required, opened a new log I0107 02:52:39.902914 4914 raft_consensus.cc:340] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151db0fe99064b19ba0d87577bb11ff6" member_type: VOTER } I0107 02:52:39.903023 4914 raft_consensus.cc:366] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:39.903087 4914 raft_consensus.cc:705] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 151db0fe99064b19ba0d87577bb11ff6, State: Initialized, Role: FOLLOWER I0107 02:52:39.903707 4914 consensus_queue.cc:231] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151db0fe99064b19ba0d87577bb11ff6" member_type: VOTER } I0107 02:52:39.903779 4914 raft_consensus.cc:380] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:39.903820 4914 raft_consensus.cc:472] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:39.903885 4914 raft_consensus.cc:2918] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:39.904870 4914 raft_consensus.cc:494] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151db0fe99064b19ba0d87577bb11ff6" member_type: VOTER } I0107 02:52:39.905462 4914 leader_election.cc:307] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 151db0fe99064b19ba0d87577bb11ff6; no voters: I0107 02:52:39.905980 4914 leader_election.cc:293] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:39.906234 4916 raft_consensus.cc:2672] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:39.906399 4916 raft_consensus.cc:667] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [term 1 LEADER]: Becoming Leader. State: Replica: 151db0fe99064b19ba0d87577bb11ff6, State: Running, Role: LEADER I0107 02:52:39.907002 4916 consensus_queue.cc:208] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151db0fe99064b19ba0d87577bb11ff6" member_type: VOTER } I0107 02:52:39.908588 4916 sys_catalog.cc:339] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 151db0fe99064b19ba0d87577bb11ff6. Latest consensus state: current_term: 1 leader_uuid: "151db0fe99064b19ba0d87577bb11ff6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151db0fe99064b19ba0d87577bb11ff6" member_type: VOTER } } I0107 02:52:39.908665 4916 sys_catalog.cc:342] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [sys.catalog]: This master's current role is: LEADER I0107 02:52:39.909040 4918 sys_catalog.cc:339] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "151db0fe99064b19ba0d87577bb11ff6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151db0fe99064b19ba0d87577bb11ff6" member_type: VOTER } } I0107 02:52:39.909067 4918 sys_catalog.cc:342] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [sys.catalog]: This master's current role is: LEADER I0107 02:52:39.909099 4914 sys_catalog.cc:339] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "151db0fe99064b19ba0d87577bb11ff6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151db0fe99064b19ba0d87577bb11ff6" member_type: VOTER } } I0107 02:52:39.909119 4914 sys_catalog.cc:342] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [sys.catalog]: This master's current role is: LEADER I0107 02:52:39.910388 4914 sys_catalog.cc:436] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [sys.catalog]: configured and running, proceeding with master startup. I0107 02:52:39.910730 2710 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 W0107 02:52:39.914597 4921 catalog_manager.cc:1158] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6: acquiring CA information for follower catalog manager: Not found: root CA entry not found W0107 02:52:39.914641 4921 catalog_manager.cc:630] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry I0107 02:52:39.914683 4920 catalog_manager.cc:1089] Loading table and tablet metadata into memory... I0107 02:52:39.915725 4920 catalog_manager.cc:1100] Initializing Kudu internal certificate authority... I0107 02:52:39.952510 4920 catalog_manager.cc:984] Generated new certificate authority record I0107 02:52:39.954090 4920 catalog_manager.cc:1109] Loading token signing keys... I0107 02:52:39.965203 4920 catalog_manager.cc:4455] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6: Generated new TSK 0 I0107 02:52:39.965903 4920 catalog_manager.cc:1119] Initializing in-progress tserver states... W0107 02:52:39.979859 2710 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters. I0107 02:52:39.979908 2710 hybrid_clock.cc:186] HybridClock initialized: now 1578365559979901 us; error 0 us; skew 500 ppm I0107 02:52:39.980039 2710 fs_manager.cc:263] Metadata directory not provided I0107 02:52:39.980078 2710 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:39.980149 2710 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:39.980162 2710 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:39.980597 2710 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/ts-0-root/instance: uuid: "48ca38702be441c3a1986c9aee01e7b3" format_stamp: "Formatted at 2020-01-07 02:52:39 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:39.981896 2710 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/ts-0-root/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/ts-0-root/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:39.986886 2710 fs_manager.cc:518] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s I0107 02:52:39.987515 2710 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:39.989368 2710 fs_manager.cc:434] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s I0107 02:52:39.989434 2710 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/ts-0-root uuid: "48ca38702be441c3a1986c9aee01e7b3" format_stamp: "Formatted at 2020-01-07 02:52:39 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:39.989483 2710 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestLocalReplicaCMetaOps.1578365549852848-2710/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:40.013386 2710 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:40.013463 2710 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:40.018502 2710 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.129:38253 I0107 02:52:40.019024 2710 webserver.cc:382] Webserver started at http://127.2.165.129:36987/ using document root and password file I0107 02:52:40.027345 4990 heartbeater.cc:322] Connected to a master server at 127.2.165.190:43261 I0107 02:52:40.027518 4990 heartbeater.cc:406] Registering TS with master... I0107 02:52:40.027904 4990 heartbeater.cc:455] Master 127.2.165.190:43261 requested a full tablet report, sending... I0107 02:52:40.029511 4876 ts_manager.cc:192] Registered new tserver with Master: 48ca38702be441c3a1986c9aee01e7b3 (127.2.165.129:38253) I0107 02:52:40.029731 2710 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.010237825s I0107 02:52:40.030372 2710 test_util.cc:229] Using random seed: -561511261 I0107 02:52:40.031296 4876 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59952 I0107 02:52:40.043176 4876 catalog_manager.cc:1454] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59954: name: "test-workload" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { } partition_schema { range_schema { columns { name: "key" } } } I0107 02:52:40.057845 4957 tablet_service.cc:1034] Processing CreateTablet for tablet 8711da64d16e4adabc184ec654c1a327 (table=test-workload [id=c993b5e32a994c7991d451871b478318]), partition=RANGE (key) PARTITION UNBOUNDED I0107 02:52:40.058317 4957 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 8711da64d16e4adabc184ec654c1a327. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:40.060952 5002 tablet_bootstrap.cc:439] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3: Bootstrap starting. I0107 02:52:40.061873 5002 tablet_bootstrap.cc:590] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3: No blocks or log segments found. Creating new log. I0107 02:52:40.062923 5002 tablet_bootstrap.cc:439] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3: No bootstrap required, opened a new log I0107 02:52:40.063022 5002 ts_tablet_manager.cc:1103] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:40.063549 5002 raft_consensus.cc:340] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ca38702be441c3a1986c9aee01e7b3" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 38253 } } I0107 02:52:40.063583 5002 raft_consensus.cc:366] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:40.063599 5002 raft_consensus.cc:705] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48ca38702be441c3a1986c9aee01e7b3, State: Initialized, Role: FOLLOWER I0107 02:52:40.063722 5002 consensus_queue.cc:231] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ca38702be441c3a1986c9aee01e7b3" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 38253 } } I0107 02:52:40.063752 5002 raft_consensus.cc:380] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:40.063766 5002 raft_consensus.cc:472] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:40.063782 5002 raft_consensus.cc:2918] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:40.064452 5002 raft_consensus.cc:494] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ca38702be441c3a1986c9aee01e7b3" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 38253 } } I0107 02:52:40.064558 5002 leader_election.cc:307] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48ca38702be441c3a1986c9aee01e7b3; no voters: I0107 02:52:40.064745 5002 leader_election.cc:293] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:40.064905 5004 raft_consensus.cc:2672] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:40.065068 5004 raft_consensus.cc:667] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [term 1 LEADER]: Becoming Leader. State: Replica: 48ca38702be441c3a1986c9aee01e7b3, State: Running, Role: LEADER I0107 02:52:40.065203 5004 consensus_queue.cc:208] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ca38702be441c3a1986c9aee01e7b3" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 38253 } } I0107 02:52:40.065297 5002 ts_tablet_manager.cc:1131] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s I0107 02:52:40.065341 4990 heartbeater.cc:447] Master 127.2.165.190:43261 was elected leader, sending a full tablet report... I0107 02:52:40.068089 4876 catalog_manager.cc:4143] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 reported cstate change: term changed from 0 to 1, leader changed from to 48ca38702be441c3a1986c9aee01e7b3 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "48ca38702be441c3a1986c9aee01e7b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ca38702be441c3a1986c9aee01e7b3" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 38253 } health_report { overall_health: HEALTHY } } } I0107 02:52:40.191412 2710 tablet_server.cc:152] TabletServer@127.2.165.129:0 shutting down... I0107 02:52:40.198033 2710 ts_tablet_manager.cc:1177] Shutting down tablet manager... I0107 02:52:40.198297 2710 tablet_replica.cc:282] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3: stopping tablet replica I0107 02:52:40.198711 2710 raft_consensus.cc:2155] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [term 1 LEADER]: Raft consensus shutting down. I0107 02:52:40.198866 2710 raft_consensus.cc:2182] T 8711da64d16e4adabc184ec654c1a327 P 48ca38702be441c3a1986c9aee01e7b3 [term 1 FOLLOWER]: Raft consensus is shut down! I0107 02:52:40.214061 2710 tablet_server.cc:166] TabletServer@127.2.165.129:0 shutdown complete. I0107 02:52:40.662241 2710 master.cc:248] Master@127.2.165.190:43261 shutting down... I0107 02:52:40.666666 2710 raft_consensus.cc:2155] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [term 1 LEADER]: Raft consensus shutting down. I0107 02:52:40.666785 2710 raft_consensus.cc:2182] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6 [term 1 FOLLOWER]: Raft consensus is shut down! I0107 02:52:40.666884 2710 tablet_replica.cc:282] T 00000000000000000000000000000000 P 151db0fe99064b19ba0d87577bb11ff6: stopping tablet replica I0107 02:52:40.679625 2710 master.cc:261] Master@127.2.165.190:43261 shutdown complete. [ OK ] ToolTest.TestLocalReplicaCMetaOps (835 ms) [ RUN ] ToolTest.TestTServerListState 2020-01-07T02:52:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG) 2020-01-07T02:52:40Z Disabled control of system clock I0107 02:52:40.693784 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data --block_manager=log --webserver_interface=localhost --ipki_ca_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:41343 --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --rpc_reuseport=true --builtin_ntp_servers=127.2.165.148:42933 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:40.761291 5044 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:40.761519 5044 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:40.762573 5044 flags.cc:404] Enabled experimental flag: --ipki_ca_key_size=1024 W0107 02:52:40.762593 5044 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:40.762600 5044 flags.cc:404] Enabled experimental flag: --tsk_num_rsa_bits=512 W0107 02:52:40.762609 5044 flags.cc:404] Enabled experimental flag: --rpc_reuseport=true I0107 02:52:40.764554 5044 master_main.cc:96] Master server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:42933 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/wal --ipki_ca_key_size=1024 --ipki_server_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:41343 --rpc_reuseport=true --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data/info.pb --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync=true --heap_profile_path=/tmp/kudu-master.5044 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/logs --logbuflevel=-1 --logtostderr=true Master server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:40.774005 5044 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:40.775213 5044 hybrid_clock.cc:186] HybridClock initialized: now 1578365560775151 us; error 54 us; skew 500 ppm I0107 02:52:40.775892 5044 fs_manager.cc:263] Metadata directory not provided I0107 02:52:40.775933 5044 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:40.776193 5044 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:40.776218 5044 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:40.778545 5044 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data/instance: uuid: "e15dfe55685d43da9b115657a24862e2" format_stamp: "Formatted at 2020-01-07 02:52:40 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:40.778808 5044 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/wal/instance: uuid: "e15dfe55685d43da9b115657a24862e2" format_stamp: "Formatted at 2020-01-07 02:52:40 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:40.780531 5044 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:40.786753 5044 fs_manager.cc:518] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s I0107 02:52:40.787555 5044 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:40.788116 5044 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:40.792590 5044 fs_manager.cc:434] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s I0107 02:52:40.792726 5044 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/wal uuid: "e15dfe55685d43da9b115657a24862e2" format_stamp: "Formatted at 2020-01-07 02:52:40 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:40.792879 5044 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:40.830106 5044 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:40.836045 5044 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.190:41343 I0107 02:52:40.837059 5044 webserver.cc:382] Webserver started at http://127.2.165.190:40939/ using document root and password file I0107 02:52:40.837741 5044 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data/info.pb I0107 02:52:40.839293 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master as pid 5044 I0107 02:52:40.839790 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data --block_manager=log --rpc_bind_addresses=127.2.165.129:0 --local_ip_for_outbound_sockets=127.2.165.129 --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:41343 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.2.165.148:42933 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} I0107 02:52:40.841224 5105 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:40.855958 5105 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2: Bootstrap starting. I0107 02:52:40.863076 5105 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2: No blocks or log segments found. Creating new log. I0107 02:52:40.864217 5105 log.cc:815] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2: Log is configured to *not* fsync() on all Append() calls I0107 02:52:40.865962 5105 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2: No bootstrap required, opened a new log I0107 02:52:40.869238 5105 raft_consensus.cc:340] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } I0107 02:52:40.869323 5105 raft_consensus.cc:366] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:40.869359 5105 raft_consensus.cc:705] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e15dfe55685d43da9b115657a24862e2, State: Initialized, Role: FOLLOWER I0107 02:52:40.870059 5105 consensus_queue.cc:231] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } I0107 02:52:40.870136 5105 raft_consensus.cc:380] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:40.870172 5105 raft_consensus.cc:472] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:40.870254 5105 raft_consensus.cc:2918] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:40.871171 5105 raft_consensus.cc:494] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } I0107 02:52:40.871654 5105 leader_election.cc:307] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e15dfe55685d43da9b115657a24862e2; no voters: I0107 02:52:40.872090 5105 leader_election.cc:293] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:40.872254 5108 raft_consensus.cc:2672] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:40.872419 5108 raft_consensus.cc:667] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 1 LEADER]: Becoming Leader. State: Replica: e15dfe55685d43da9b115657a24862e2, State: Running, Role: LEADER I0107 02:52:40.872805 5108 consensus_queue.cc:208] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } I0107 02:52:40.874912 5109 sys_catalog.cc:339] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e15dfe55685d43da9b115657a24862e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } } I0107 02:52:40.874912 5110 sys_catalog.cc:339] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e15dfe55685d43da9b115657a24862e2. Latest consensus state: current_term: 1 leader_uuid: "e15dfe55685d43da9b115657a24862e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } } I0107 02:52:40.874960 5110 sys_catalog.cc:342] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: This master's current role is: LEADER I0107 02:52:40.874960 5109 sys_catalog.cc:342] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: This master's current role is: LEADER I0107 02:52:40.875264 5105 sys_catalog.cc:339] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "e15dfe55685d43da9b115657a24862e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } } I0107 02:52:40.875298 5105 sys_catalog.cc:342] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: This master's current role is: LEADER I0107 02:52:40.876380 5105 sys_catalog.cc:436] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: configured and running, proceeding with master startup. I0107 02:52:40.876610 5112 catalog_manager.cc:1089] Loading table and tablet metadata into memory... I0107 02:52:40.885236 5112 catalog_manager.cc:1100] Initializing Kudu internal certificate authority... WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:40.921334 5106 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:40.921639 5106 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:40.923033 5106 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:40.923131 5106 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.165.129 I0107 02:52:40.925865 5106 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:42933 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.2.165.129:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:41343 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.5106 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.2.165.129 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:40.938591 5106 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:40.944535 5112 catalog_manager.cc:984] Generated new certificate authority record I0107 02:52:40.945703 5112 catalog_manager.cc:1109] Loading token signing keys... I0107 02:52:40.946115 5106 hybrid_clock.cc:186] HybridClock initialized: now 1578365560946049 us; error 40 us; skew 500 ppm I0107 02:52:40.947055 5106 fs_manager.cc:263] Metadata directory not provided I0107 02:52:40.947124 5106 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:40.947458 5106 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:40.947494 5106 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:40.950608 5106 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data/instance: uuid: "da155ca1c6f34a779b3bd550f46801b0" format_stamp: "Formatted at 2020-01-07 02:52:40 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:40.950987 5106 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/wal/instance: uuid: "da155ca1c6f34a779b3bd550f46801b0" format_stamp: "Formatted at 2020-01-07 02:52:40 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:40.953493 5106 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:40.959079 5112 catalog_manager.cc:4455] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2: Generated new TSK 0 I0107 02:52:40.959705 5112 catalog_manager.cc:1119] Initializing in-progress tserver states... I0107 02:52:40.963910 5106 fs_manager.cc:518] Time spent creating directory manager: real 0.013s user 0.010s sys 0.000s I0107 02:52:40.964860 5106 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:40.965487 5106 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:40.970736 5106 fs_manager.cc:434] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s I0107 02:52:40.970896 5106 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/wal uuid: "da155ca1c6f34a779b3bd550f46801b0" format_stamp: "Formatted at 2020-01-07 02:52:40 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:40.971055 5106 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:41.000744 5106 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:41.002362 5106 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:41.002400 5106 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:41.013015 5106 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.129:40435 I0107 02:52:41.014287 5106 webserver.cc:382] Webserver started at http://127.2.165.129:41205/ using document root and password file I0107 02:52:41.015098 5106 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data/info.pb I0107 02:52:41.017731 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 5106 I0107 02:52:41.027649 5237 heartbeater.cc:322] Connected to a master server at 127.2.165.190:41343 I0107 02:52:41.027832 5237 heartbeater.cc:406] Registering TS with master... I0107 02:52:41.028338 5237 heartbeater.cc:455] Master 127.2.165.190:41343 requested a full tablet report, sending... I0107 02:52:41.029999 5072 ts_manager.cc:192] Registered new tserver with Master: da155ca1c6f34a779b3bd550f46801b0 (127.2.165.129:40435) I0107 02:52:41.032032 2710 external_mini_cluster.cc:721] 1 TS(s) registered with all masters I0107 02:52:41.032119 5072 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 127.2.165.129:39547 I0107 02:52:41.160631 5071 ts_manager.cc:278] Set tserver state for da155ca1c6f34a779b3bd550f46801b0 to MAINTENANCE_MODE I0107 02:52:41.577864 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 5106 I0107 02:52:41.581328 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master with pid 5044 I0107 02:52:41.585127 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data --block_manager=log --webserver_interface=localhost --ipki_ca_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:41343 --webserver_interface=127.2.165.190 --webserver_port=40939 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --rpc_reuseport=true --builtin_ntp_servers=127.2.165.148:42933 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:41.652551 5296 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:41.652782 5296 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:41.653836 5296 flags.cc:404] Enabled experimental flag: --ipki_ca_key_size=1024 W0107 02:52:41.653862 5296 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:41.653872 5296 flags.cc:404] Enabled experimental flag: --tsk_num_rsa_bits=512 W0107 02:52:41.653884 5296 flags.cc:404] Enabled experimental flag: --rpc_reuseport=true I0107 02:52:41.655844 5296 master_main.cc:96] Master server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:42933 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/wal --ipki_ca_key_size=1024 --ipki_server_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:41343 --rpc_reuseport=true --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data/info.pb --webserver_interface=127.2.165.190 --webserver_port=40939 --never_fsync=true --heap_profile_path=/tmp/kudu-master.5296 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/logs --logbuflevel=-1 --logtostderr=true Master server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:41.664778 5296 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:41.665977 5296 hybrid_clock.cc:186] HybridClock initialized: now 1578365561665922 us; error 46 us; skew 500 ppm I0107 02:52:41.666712 5296 fs_manager.cc:263] Metadata directory not provided I0107 02:52:41.666776 5296 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:41.673339 5296 fs_manager.cc:414] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s I0107 02:52:41.673878 5296 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:41.674365 5296 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:41.678242 5296 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:41.678963 5296 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/wal uuid: "e15dfe55685d43da9b115657a24862e2" format_stamp: "Formatted at 2020-01-07 02:52:40 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:41.679116 5296 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:41.720162 5296 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:41.725591 5296 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.190:41343 I0107 02:52:41.726927 5296 webserver.cc:382] Webserver started at http://127.2.165.190:40939/ using document root and password file I0107 02:52:41.727703 5296 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/master-0/data/info.pb I0107 02:52:41.731088 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master as pid 5296 I0107 02:52:41.733763 5357 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2: Bootstrap starting. I0107 02:52:41.764580 5357 log.cc:815] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2: Log is configured to *not* fsync() on all Append() calls I0107 02:52:41.772758 5357 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates I0107 02:52:41.773077 5357 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2: Bootstrap complete. I0107 02:52:41.776912 5357 raft_consensus.cc:340] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } I0107 02:52:41.777024 5357 raft_consensus.cc:705] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e15dfe55685d43da9b115657a24862e2, State: Initialized, Role: FOLLOWER I0107 02:52:41.777534 5357 consensus_queue.cc:231] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } I0107 02:52:41.777601 5357 raft_consensus.cc:380] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:41.777648 5357 raft_consensus.cc:472] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:41.777732 5357 raft_consensus.cc:2918] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 1 FOLLOWER]: Advancing to term 2 I0107 02:52:41.779469 5357 raft_consensus.cc:494] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } I0107 02:52:41.779901 5357 leader_election.cc:307] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e15dfe55685d43da9b115657a24862e2; no voters: I0107 02:52:41.780340 5357 leader_election.cc:293] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [CANDIDATE]: Term 2 election: Requested vote from peers I0107 02:52:41.780668 5357 sys_catalog.cc:339] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } } I0107 02:52:41.780701 5357 sys_catalog.cc:342] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: This master's current role is: FOLLOWER I0107 02:52:41.781425 5357 sys_catalog.cc:436] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: configured and running, proceeding with master startup. I0107 02:52:41.781785 5360 raft_consensus.cc:2672] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 2 FOLLOWER]: Leader election won for term 2 I0107 02:52:41.782115 5360 raft_consensus.cc:667] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [term 2 LEADER]: Becoming Leader. State: Replica: e15dfe55685d43da9b115657a24862e2, State: Running, Role: LEADER I0107 02:52:41.782631 5360 consensus_queue.cc:208] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } I0107 02:52:41.784215 5363 sys_catalog.cc:339] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e15dfe55685d43da9b115657a24862e2. Latest consensus state: current_term: 2 leader_uuid: "e15dfe55685d43da9b115657a24862e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } } I0107 02:52:41.784292 5363 sys_catalog.cc:342] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: This master's current role is: LEADER I0107 02:52:41.784632 5361 sys_catalog.cc:339] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e15dfe55685d43da9b115657a24862e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15dfe55685d43da9b115657a24862e2" member_type: VOTER } } I0107 02:52:41.784660 5361 sys_catalog.cc:342] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2 [sys.catalog]: This master's current role is: LEADER I0107 02:52:41.792974 5362 catalog_manager.cc:1156] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2: acquiring CA information for follower catalog manager: success I0107 02:52:41.794307 5362 catalog_manager.cc:1184] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0 I0107 02:52:41.794440 5365 catalog_manager.cc:1089] Loading table and tablet metadata into memory... I0107 02:52:41.795712 5365 catalog_manager.cc:1100] Initializing Kudu internal certificate authority... I0107 02:52:41.796533 5365 catalog_manager.cc:1109] Loading token signing keys... I0107 02:52:41.797067 5365 catalog_manager.cc:4466] T 00000000000000000000000000000000 P e15dfe55685d43da9b115657a24862e2: Loaded TSK: 0 I0107 02:52:41.797726 5365 catalog_manager.cc:1119] Initializing in-progress tserver states... I0107 02:52:42.155819 5322 ts_manager.cc:267] Unset tserver state for da155ca1c6f34a779b3bd550f46801b0 from MAINTENANCE_MODE I0107 02:52:42.292567 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data --block_manager=log --rpc_bind_addresses=127.2.165.129:40435 --local_ip_for_outbound_sockets=127.2.165.129 --webserver_port=41205 --webserver_interface=127.2.165.129 --tserver_master_addrs=127.2.165.190:41343 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.2.165.148:42933 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:42.347523 5420 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:42.347733 5420 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:42.348645 5420 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:42.348706 5420 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.165.129 I0107 02:52:42.350361 5420 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:42933 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.2.165.129:40435 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --webserver_interface=127.2.165.129 --webserver_port=41205 --tserver_master_addrs=127.2.165.190:41343 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.5420 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.2.165.129 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:42.359690 5420 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:42.360874 5420 hybrid_clock.cc:186] HybridClock initialized: now 1578365562360818 us; error 52 us; skew 500 ppm I0107 02:52:42.361564 5420 fs_manager.cc:263] Metadata directory not provided I0107 02:52:42.361626 5420 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:42.368124 5420 fs_manager.cc:414] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s I0107 02:52:42.368635 5420 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:42.369204 5420 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:42.373337 5420 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:42.374112 5420 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/wal uuid: "da155ca1c6f34a779b3bd550f46801b0" format_stamp: "Formatted at 2020-01-07 02:52:40 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:42.374269 5420 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:42.428144 5420 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:42.429638 5420 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:42.429669 5420 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:42.437685 5420 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.129:40435 I0107 02:52:42.438725 5420 webserver.cc:382] Webserver started at http://127.2.165.129:41205/ using document root and password file I0107 02:52:42.439425 5420 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestTServerListState.1578365549852848-2710/minicluster-data/ts-0/data/info.pb I0107 02:52:42.448101 5541 heartbeater.cc:322] Connected to a master server at 127.2.165.190:41343 I0107 02:52:42.448276 5541 heartbeater.cc:406] Registering TS with master... I0107 02:52:42.448794 5541 heartbeater.cc:455] Master 127.2.165.190:41343 requested a full tablet report, sending... I0107 02:52:42.448830 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 5420 I0107 02:52:42.450270 5322 ts_manager.cc:192] Registered new tserver with Master: da155ca1c6f34a779b3bd550f46801b0 (127.2.165.129:40435) I0107 02:52:42.452154 5322 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 127.2.165.129:55981 I0107 02:52:42.735152 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 5420 I0107 02:52:42.738617 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master with pid 5296 2020-01-07T02:52:42Z chronyd exiting [ OK ] ToolTest.TestTServerListState (2070 ms) [ RUN ] ToolTest.TestRenameColumn 2020-01-07T02:52:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG) 2020-01-07T02:52:42Z Disabled control of system clock I0107 02:52:42.764000 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/data --block_manager=log --webserver_interface=localhost --ipki_ca_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:38279 --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --rpc_reuseport=true --builtin_ntp_servers=127.2.165.148:35011 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:42.828996 5580 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:42.829218 5580 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:42.830305 5580 flags.cc:404] Enabled experimental flag: --ipki_ca_key_size=1024 W0107 02:52:42.830329 5580 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:42.830337 5580 flags.cc:404] Enabled experimental flag: --tsk_num_rsa_bits=512 W0107 02:52:42.830345 5580 flags.cc:404] Enabled experimental flag: --rpc_reuseport=true I0107 02:52:42.832343 5580 master_main.cc:96] Master server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:35011 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/wal --ipki_ca_key_size=1024 --ipki_server_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:38279 --rpc_reuseport=true --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/data/info.pb --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync=true --heap_profile_path=/tmp/kudu-master.5580 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/logs --logbuflevel=-1 --logtostderr=true Master server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:42.841567 5580 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:42.842754 5580 hybrid_clock.cc:186] HybridClock initialized: now 1578365562842724 us; error 25 us; skew 500 ppm I0107 02:52:42.843510 5580 fs_manager.cc:263] Metadata directory not provided I0107 02:52:42.843557 5580 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:42.843842 5580 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:42.843870 5580 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:42.846174 5580 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/data/instance: uuid: "e640d927ad584bd494574a7d84531c50" format_stamp: "Formatted at 2020-01-07 02:52:42 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:42.846513 5580 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/wal/instance: uuid: "e640d927ad584bd494574a7d84531c50" format_stamp: "Formatted at 2020-01-07 02:52:42 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:42.848210 5580 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:42.854100 5580 fs_manager.cc:518] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s I0107 02:52:42.854885 5580 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:42.855402 5580 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:42.859427 5580 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s I0107 02:52:42.859539 5580 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/wal uuid: "e640d927ad584bd494574a7d84531c50" format_stamp: "Formatted at 2020-01-07 02:52:42 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:42.859683 5580 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:42.913125 5580 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:42.919826 5580 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.190:38279 I0107 02:52:42.920831 5580 webserver.cc:382] Webserver started at http://127.2.165.190:32773/ using document root and password file I0107 02:52:42.921486 5580 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/master-0/data/info.pb I0107 02:52:42.924930 5641 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:42.929579 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master as pid 5580 I0107 02:52:42.930131 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/data --block_manager=log --rpc_bind_addresses=127.2.165.129:0 --local_ip_for_outbound_sockets=127.2.165.129 --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:38279 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.2.165.148:35011 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} I0107 02:52:42.930598 5641 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50: Bootstrap starting. I0107 02:52:42.933061 5641 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50: No blocks or log segments found. Creating new log. I0107 02:52:42.934145 5641 log.cc:815] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50: Log is configured to *not* fsync() on all Append() calls I0107 02:52:42.935815 5641 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50: No bootstrap required, opened a new log I0107 02:52:42.938895 5641 raft_consensus.cc:340] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e640d927ad584bd494574a7d84531c50" member_type: VOTER } I0107 02:52:42.938989 5641 raft_consensus.cc:366] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:42.939046 5641 raft_consensus.cc:705] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e640d927ad584bd494574a7d84531c50, State: Initialized, Role: FOLLOWER I0107 02:52:42.939585 5641 consensus_queue.cc:231] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e640d927ad584bd494574a7d84531c50" member_type: VOTER } I0107 02:52:42.939656 5641 raft_consensus.cc:380] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:42.939718 5641 raft_consensus.cc:472] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:42.939795 5641 raft_consensus.cc:2918] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:42.940567 5641 raft_consensus.cc:494] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e640d927ad584bd494574a7d84531c50" member_type: VOTER } I0107 02:52:42.941028 5641 leader_election.cc:307] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e640d927ad584bd494574a7d84531c50; no voters: I0107 02:52:42.941455 5641 leader_election.cc:293] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:42.941660 5644 raft_consensus.cc:2672] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:42.941865 5644 raft_consensus.cc:667] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [term 1 LEADER]: Becoming Leader. State: Replica: e640d927ad584bd494574a7d84531c50, State: Running, Role: LEADER I0107 02:52:42.942241 5644 consensus_queue.cc:208] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e640d927ad584bd494574a7d84531c50" member_type: VOTER } I0107 02:52:42.944010 5644 sys_catalog.cc:339] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e640d927ad584bd494574a7d84531c50. Latest consensus state: current_term: 1 leader_uuid: "e640d927ad584bd494574a7d84531c50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e640d927ad584bd494574a7d84531c50" member_type: VOTER } } I0107 02:52:42.944519 5641 sys_catalog.cc:339] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "e640d927ad584bd494574a7d84531c50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e640d927ad584bd494574a7d84531c50" member_type: VOTER } } I0107 02:52:42.945482 5641 sys_catalog.cc:342] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [sys.catalog]: This master's current role is: LEADER I0107 02:52:42.945453 5644 sys_catalog.cc:342] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [sys.catalog]: This master's current role is: LEADER I0107 02:52:42.944519 5646 sys_catalog.cc:339] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e640d927ad584bd494574a7d84531c50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e640d927ad584bd494574a7d84531c50" member_type: VOTER } } I0107 02:52:42.945600 5646 sys_catalog.cc:342] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [sys.catalog]: This master's current role is: LEADER I0107 02:52:42.946619 5641 sys_catalog.cc:436] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50 [sys.catalog]: configured and running, proceeding with master startup. I0107 02:52:42.946775 5648 catalog_manager.cc:1089] Loading table and tablet metadata into memory... I0107 02:52:42.950356 5648 catalog_manager.cc:1100] Initializing Kudu internal certificate authority... I0107 02:52:43.007457 5648 catalog_manager.cc:984] Generated new certificate authority record I0107 02:52:43.008574 5648 catalog_manager.cc:1109] Loading token signing keys... I0107 02:52:43.017786 5648 catalog_manager.cc:4455] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50: Generated new TSK 0 I0107 02:52:43.018359 5648 catalog_manager.cc:1119] Initializing in-progress tserver states... WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:43.049337 5642 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:43.049569 5642 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:43.054615 5642 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:43.054694 5642 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.165.129 I0107 02:52:43.056619 5642 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:35011 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.2.165.129:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:38279 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.5642 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.2.165.129 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:43.078900 5642 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:43.080178 5642 hybrid_clock.cc:186] HybridClock initialized: now 1578365563080122 us; error 41 us; skew 500 ppm I0107 02:52:43.081182 5642 fs_manager.cc:263] Metadata directory not provided I0107 02:52:43.081275 5642 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:43.081581 5642 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:43.081614 5642 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:43.084008 5642 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/data/instance: uuid: "a78054d27c44481c9aa1a70bce4d15dd" format_stamp: "Formatted at 2020-01-07 02:52:43 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:43.084331 5642 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/wal/instance: uuid: "a78054d27c44481c9aa1a70bce4d15dd" format_stamp: "Formatted at 2020-01-07 02:52:43 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:43.086062 5642 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:43.093444 5642 fs_manager.cc:518] Time spent creating directory manager: real 0.009s user 0.009s sys 0.001s I0107 02:52:43.094223 5642 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:43.094806 5642 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:43.098703 5642 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s I0107 02:52:43.098860 5642 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/wal uuid: "a78054d27c44481c9aa1a70bce4d15dd" format_stamp: "Formatted at 2020-01-07 02:52:43 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:43.099005 5642 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:43.151830 5642 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:43.153209 5642 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:43.153242 5642 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:43.162075 5642 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.129:43475 I0107 02:52:43.163096 5642 webserver.cc:382] Webserver started at http://127.2.165.129:36817/ using document root and password file I0107 02:52:43.163672 5642 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestRenameColumn.1578365549852848-2710/minicluster-data/ts-0/data/info.pb I0107 02:52:43.168099 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 5642 I0107 02:52:43.176326 5773 heartbeater.cc:322] Connected to a master server at 127.2.165.190:38279 I0107 02:52:43.176512 5773 heartbeater.cc:406] Registering TS with master... I0107 02:52:43.176993 5773 heartbeater.cc:455] Master 127.2.165.190:38279 requested a full tablet report, sending... I0107 02:52:43.178503 5608 ts_manager.cc:192] Registered new tserver with Master: a78054d27c44481c9aa1a70bce4d15dd (127.2.165.129:43475) I0107 02:52:43.180040 2710 external_mini_cluster.cc:721] 1 TS(s) registered with all masters I0107 02:52:43.180089 5608 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 127.2.165.129:43617 I0107 02:52:43.181635 2710 test_util.cc:229] Using random seed: -558360010 I0107 02:52:43.191713 5607 catalog_manager.cc:1454] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36842: name: "table" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "col.0" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { } partition_schema { range_schema { columns { name: "key" } } } I0107 02:52:43.208653 5704 tablet_service.cc:1034] Processing CreateTablet for tablet 8fbf7cc397734cfaa6f95732bb45ca52 (table=table [id=b3ab6c195f754adcbb43a6b7b392fe68]), partition=RANGE (key) PARTITION UNBOUNDED I0107 02:52:43.209271 5704 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 8fbf7cc397734cfaa6f95732bb45ca52. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:43.214784 5786 tablet_bootstrap.cc:439] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd: Bootstrap starting. I0107 02:52:43.217052 5786 tablet_bootstrap.cc:590] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd: No blocks or log segments found. Creating new log. I0107 02:52:43.217985 5786 log.cc:815] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd: Log is configured to *not* fsync() on all Append() calls I0107 02:52:43.219282 5786 tablet_bootstrap.cc:439] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd: No bootstrap required, opened a new log I0107 02:52:43.219517 5786 ts_tablet_manager.cc:1103] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s I0107 02:52:43.222055 5786 raft_consensus.cc:340] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a78054d27c44481c9aa1a70bce4d15dd" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 43475 } } I0107 02:52:43.222151 5786 raft_consensus.cc:366] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:43.222200 5786 raft_consensus.cc:705] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a78054d27c44481c9aa1a70bce4d15dd, State: Initialized, Role: FOLLOWER I0107 02:52:43.222713 5786 consensus_queue.cc:231] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a78054d27c44481c9aa1a70bce4d15dd" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 43475 } } I0107 02:52:43.222781 5786 raft_consensus.cc:380] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:43.222831 5786 raft_consensus.cc:472] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:43.222924 5786 raft_consensus.cc:2918] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:43.223759 5786 raft_consensus.cc:494] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a78054d27c44481c9aa1a70bce4d15dd" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 43475 } } I0107 02:52:43.224189 5786 leader_election.cc:307] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a78054d27c44481c9aa1a70bce4d15dd; no voters: I0107 02:52:43.224517 5786 leader_election.cc:293] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:43.225005 5773 heartbeater.cc:447] Master 127.2.165.190:38279 was elected leader, sending a full tablet report... I0107 02:52:43.225273 5786 ts_tablet_manager.cc:1131] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s I0107 02:52:43.225575 5788 raft_consensus.cc:2672] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:43.225740 5788 raft_consensus.cc:667] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd [term 1 LEADER]: Becoming Leader. State: Replica: a78054d27c44481c9aa1a70bce4d15dd, State: Running, Role: LEADER I0107 02:52:43.226023 5788 consensus_queue.cc:208] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a78054d27c44481c9aa1a70bce4d15dd" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 43475 } } I0107 02:52:43.229300 5607 catalog_manager.cc:4143] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd reported cstate change: term changed from 0 to 1, leader changed from to a78054d27c44481c9aa1a70bce4d15dd (127.2.165.129). New cstate: current_term: 1 leader_uuid: "a78054d27c44481c9aa1a70bce4d15dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a78054d27c44481c9aa1a70bce4d15dd" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 43475 } health_report { overall_health: HEALTHY } } } I0107 02:52:43.360369 5607 catalog_manager.cc:2379] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36846: table { table_name: "table" } alter_schema_steps { type: RENAME_COLUMN rename_column { old_name: "col.0" new_name: "col_0" } } modify_external_catalogs: true I0107 02:52:43.367036 5803 tablet.cc:1251] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd: Alter schema from ( 10:key INT32 NOT NULL, 11:col.0 INT32 NOT NULL, PRIMARY KEY (key) ) version 0 to ( 10:key INT32 NOT NULL, 11:col_0 INT32 NOT NULL, PRIMARY KEY (key) ) version 1 I0107 02:52:43.367941 5803 tablet.cc:1190] T 8fbf7cc397734cfaa6f95732bb45ca52 P a78054d27c44481c9aa1a70bce4d15dd: MemRowSet was empty: no flush needed. I0107 02:52:43.373633 5607 catalog_manager.cc:4600] T 00000000000000000000000000000000 P e640d927ad584bd494574a7d84531c50: table [id=b3ab6c195f754adcbb43a6b7b392fe68] alter complete (version 1) I0107 02:52:43.393399 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 5642 I0107 02:52:43.397217 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master with pid 5580 2020-01-07T02:52:43Z chronyd exiting [ OK ] ToolTest.TestRenameColumn (659 ms) [ RUN ] ToolTest.TestScanTableMultiPredicates 2020-01-07T02:52:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG) 2020-01-07T02:52:43Z Disabled control of system clock I0107 02:52:43.422937 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/data --block_manager=log --webserver_interface=localhost --ipki_ca_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:36653 --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --rpc_reuseport=true --builtin_ntp_servers=127.2.165.148:37479 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:43.491295 5812 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:43.491524 5812 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:43.492579 5812 flags.cc:404] Enabled experimental flag: --ipki_ca_key_size=1024 W0107 02:52:43.492605 5812 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:43.492611 5812 flags.cc:404] Enabled experimental flag: --tsk_num_rsa_bits=512 W0107 02:52:43.492621 5812 flags.cc:404] Enabled experimental flag: --rpc_reuseport=true I0107 02:52:43.494663 5812 master_main.cc:96] Master server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:37479 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/wal --ipki_ca_key_size=1024 --ipki_server_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:36653 --rpc_reuseport=true --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/data/info.pb --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync=true --heap_profile_path=/tmp/kudu-master.5812 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/logs --logbuflevel=-1 --logtostderr=true Master server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:43.503690 5812 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:43.504946 5812 hybrid_clock.cc:186] HybridClock initialized: now 1578365563504902 us; error 34 us; skew 500 ppm I0107 02:52:43.505648 5812 fs_manager.cc:263] Metadata directory not provided I0107 02:52:43.505694 5812 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:43.505964 5812 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:43.505990 5812 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:43.508337 5812 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/data/instance: uuid: "49129ddae4924e7cbfb0915eecb2e05a" format_stamp: "Formatted at 2020-01-07 02:52:43 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:43.508618 5812 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/wal/instance: uuid: "49129ddae4924e7cbfb0915eecb2e05a" format_stamp: "Formatted at 2020-01-07 02:52:43 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:43.510309 5812 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:43.516104 5812 fs_manager.cc:518] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s I0107 02:52:43.516861 5812 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:43.517354 5812 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:43.521294 5812 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:43.521410 5812 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/wal uuid: "49129ddae4924e7cbfb0915eecb2e05a" format_stamp: "Formatted at 2020-01-07 02:52:43 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:43.521525 5812 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:43.566933 5812 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:43.572244 5812 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.190:36653 I0107 02:52:43.573194 5812 webserver.cc:382] Webserver started at http://127.2.165.190:39699/ using document root and password file I0107 02:52:43.573845 5812 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/master-0/data/info.pb I0107 02:52:43.577193 5873 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:43.578312 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master as pid 5812 I0107 02:52:43.578851 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/data --block_manager=log --rpc_bind_addresses=127.2.165.129:0 --local_ip_for_outbound_sockets=127.2.165.129 --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:36653 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.2.165.148:37479 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} I0107 02:52:43.583886 5873 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a: Bootstrap starting. I0107 02:52:43.586530 5873 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a: No blocks or log segments found. Creating new log. I0107 02:52:43.587563 5873 log.cc:815] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a: Log is configured to *not* fsync() on all Append() calls I0107 02:52:43.589045 5873 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a: No bootstrap required, opened a new log I0107 02:52:43.592185 5873 raft_consensus.cc:340] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49129ddae4924e7cbfb0915eecb2e05a" member_type: VOTER } I0107 02:52:43.592265 5873 raft_consensus.cc:366] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:43.592312 5873 raft_consensus.cc:705] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49129ddae4924e7cbfb0915eecb2e05a, State: Initialized, Role: FOLLOWER I0107 02:52:43.592872 5873 consensus_queue.cc:231] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49129ddae4924e7cbfb0915eecb2e05a" member_type: VOTER } I0107 02:52:43.592931 5873 raft_consensus.cc:380] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:43.592977 5873 raft_consensus.cc:472] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:43.593055 5873 raft_consensus.cc:2918] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:43.593900 5873 raft_consensus.cc:494] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49129ddae4924e7cbfb0915eecb2e05a" member_type: VOTER } I0107 02:52:43.594372 5873 leader_election.cc:307] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49129ddae4924e7cbfb0915eecb2e05a; no voters: I0107 02:52:43.594851 5873 leader_election.cc:293] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:43.595036 5876 raft_consensus.cc:2672] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:43.595261 5876 raft_consensus.cc:667] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [term 1 LEADER]: Becoming Leader. State: Replica: 49129ddae4924e7cbfb0915eecb2e05a, State: Running, Role: LEADER I0107 02:52:43.595623 5876 consensus_queue.cc:208] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49129ddae4924e7cbfb0915eecb2e05a" member_type: VOTER } I0107 02:52:43.596971 5876 sys_catalog.cc:339] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49129ddae4924e7cbfb0915eecb2e05a. Latest consensus state: current_term: 1 leader_uuid: "49129ddae4924e7cbfb0915eecb2e05a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49129ddae4924e7cbfb0915eecb2e05a" member_type: VOTER } } I0107 02:52:43.597016 5876 sys_catalog.cc:342] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [sys.catalog]: This master's current role is: LEADER I0107 02:52:43.597271 5876 sys_catalog.cc:339] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49129ddae4924e7cbfb0915eecb2e05a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49129ddae4924e7cbfb0915eecb2e05a" member_type: VOTER } } I0107 02:52:43.597296 5876 sys_catalog.cc:342] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [sys.catalog]: This master's current role is: LEADER I0107 02:52:43.597362 5873 sys_catalog.cc:339] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "49129ddae4924e7cbfb0915eecb2e05a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49129ddae4924e7cbfb0915eecb2e05a" member_type: VOTER } } I0107 02:52:43.597385 5873 sys_catalog.cc:342] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [sys.catalog]: This master's current role is: LEADER I0107 02:52:43.598520 5873 sys_catalog.cc:436] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a [sys.catalog]: configured and running, proceeding with master startup. I0107 02:52:43.598649 5880 catalog_manager.cc:1089] Loading table and tablet metadata into memory... I0107 02:52:43.601720 5880 catalog_manager.cc:1100] Initializing Kudu internal certificate authority... I0107 02:52:43.633329 5880 catalog_manager.cc:984] Generated new certificate authority record I0107 02:52:43.634475 5880 catalog_manager.cc:1109] Loading token signing keys... I0107 02:52:43.641366 5880 catalog_manager.cc:4455] T 00000000000000000000000000000000 P 49129ddae4924e7cbfb0915eecb2e05a: Generated new TSK 0 I0107 02:52:43.641889 5880 catalog_manager.cc:1119] Initializing in-progress tserver states... WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:52:43.652030 5874 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:52:43.652318 5874 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:52:43.653641 5874 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:52:43.653735 5874 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.165.129 I0107 02:52:43.656100 5874 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:37479 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.2.165.129:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:36653 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.5874 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.2.165.129 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:52:43.668331 5874 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:52:43.669535 5874 hybrid_clock.cc:186] HybridClock initialized: now 1578365563669492 us; error 31 us; skew 500 ppm I0107 02:52:43.670343 5874 fs_manager.cc:263] Metadata directory not provided I0107 02:52:43.670440 5874 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:52:43.670764 5874 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:52:43.670799 5874 server_base.cc:453] Attempting to create new FS layout instead I0107 02:52:43.673494 5874 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/data/instance: uuid: "c0cb808e128744359524338fc6ce58b3" format_stamp: "Formatted at 2020-01-07 02:52:43 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:43.673822 5874 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/wal/instance: uuid: "c0cb808e128744359524338fc6ce58b3" format_stamp: "Formatted at 2020-01-07 02:52:43 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:43.676259 5874 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:52:43.687882 5874 fs_manager.cc:518] Time spent creating directory manager: real 0.014s user 0.010s sys 0.002s I0107 02:52:43.688841 5874 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:52:43.689448 5874 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:52:43.694258 5874 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:52:43.694461 5874 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/wal uuid: "c0cb808e128744359524338fc6ce58b3" format_stamp: "Formatted at 2020-01-07 02:52:43 on dist-test-slave-dist-test-slave-3kpx" I0107 02:52:43.694597 5874 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:52:43.736083 5874 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:52:43.737870 5874 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:52:43.737913 5874 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:52:43.746868 5874 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.129:46463 I0107 02:52:43.748066 5874 webserver.cc:382] Webserver started at http://127.2.165.129:41139/ using document root and password file I0107 02:52:43.748812 5874 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestScanTableMultiPredicates.1578365549852848-2710/minicluster-data/ts-0/data/info.pb I0107 02:52:43.757772 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 5874 I0107 02:52:43.764976 6005 heartbeater.cc:322] Connected to a master server at 127.2.165.190:36653 I0107 02:52:43.765187 6005 heartbeater.cc:406] Registering TS with master... I0107 02:52:43.765724 6005 heartbeater.cc:455] Master 127.2.165.190:36653 requested a full tablet report, sending... I0107 02:52:43.767343 5839 ts_manager.cc:192] Registered new tserver with Master: c0cb808e128744359524338fc6ce58b3 (127.2.165.129:46463) I0107 02:52:43.768177 2710 external_mini_cluster.cc:721] 1 TS(s) registered with all masters I0107 02:52:43.768893 5839 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 127.2.165.129:56387 I0107 02:52:43.769452 2710 test_util.cc:229] Using random seed: -557772182 I0107 02:52:43.782639 5839 catalog_manager.cc:1454] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45080: name: "kudu.table.scan.multipredicates" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { } partition_schema { range_schema { columns { name: "key" } } } I0107 02:52:43.798907 5942 tablet_service.cc:1034] Processing CreateTablet for tablet 9c35fe52f4ff455ebb68884502590215 (table=kudu.table.scan.multipredicates [id=6359fdf8ec7e4d5b9ca5e09f2c321bd9]), partition=RANGE (key) PARTITION UNBOUNDED I0107 02:52:43.799597 5942 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 9c35fe52f4ff455ebb68884502590215. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:52:43.805725 6018 tablet_bootstrap.cc:439] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3: Bootstrap starting. I0107 02:52:43.807739 6018 tablet_bootstrap.cc:590] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3: No blocks or log segments found. Creating new log. I0107 02:52:43.808712 6018 log.cc:815] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3: Log is configured to *not* fsync() on all Append() calls I0107 02:52:43.809973 6018 tablet_bootstrap.cc:439] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3: No bootstrap required, opened a new log I0107 02:52:43.810148 6018 ts_tablet_manager.cc:1103] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s I0107 02:52:43.812927 6018 raft_consensus.cc:340] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0cb808e128744359524338fc6ce58b3" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 46463 } } I0107 02:52:43.813010 6018 raft_consensus.cc:366] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:52:43.813076 6018 raft_consensus.cc:705] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0cb808e128744359524338fc6ce58b3, State: Initialized, Role: FOLLOWER I0107 02:52:43.813612 6018 consensus_queue.cc:231] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0cb808e128744359524338fc6ce58b3" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 46463 } } I0107 02:52:43.813683 6018 raft_consensus.cc:380] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:52:43.813781 6018 raft_consensus.cc:472] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:52:43.813853 6018 raft_consensus.cc:2918] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:52:43.814803 6018 raft_consensus.cc:494] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0cb808e128744359524338fc6ce58b3" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 46463 } } I0107 02:52:43.815193 6018 leader_election.cc:307] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0cb808e128744359524338fc6ce58b3; no voters: I0107 02:52:43.815524 6018 leader_election.cc:293] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:52:43.815707 6020 raft_consensus.cc:2672] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:52:43.815886 6020 raft_consensus.cc:667] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 [term 1 LEADER]: Becoming Leader. State: Replica: c0cb808e128744359524338fc6ce58b3, State: Running, Role: LEADER I0107 02:52:43.816027 6005 heartbeater.cc:447] Master 127.2.165.190:36653 was elected leader, sending a full tablet report... I0107 02:52:43.816339 6020 consensus_queue.cc:208] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0cb808e128744359524338fc6ce58b3" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 46463 } } I0107 02:52:43.816378 6018 ts_tablet_manager.cc:1131] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s I0107 02:52:43.820092 5839 catalog_manager.cc:4143] T 9c35fe52f4ff455ebb68884502590215 P c0cb808e128744359524338fc6ce58b3 reported cstate change: term changed from 0 to 1, leader changed from to c0cb808e128744359524338fc6ce58b3 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "c0cb808e128744359524338fc6ce58b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0cb808e128744359524338fc6ce58b3" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 46463 } health_report { overall_health: HEALTHY } } } I0107 02:52:44.205171 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver with pid 5874 I0107 02:52:44.209931 2710 external_mini_cluster.cc:1241] Killing /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master with pid 5812 2020-01-07T02:52:44Z chronyd exiting [ OK ] ToolTest.TestScanTableMultiPredicates (813 ms) [ RUN ] ToolTest.TestHmsIgnoresDifferentMasters 2020-01-07T02:52:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG) 2020-01-07T02:52:44Z Disabled control of system clock Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/derby.log SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/tmp/dist-test-taskh2U6sE/thirdparty/src/hive-3.1.1/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/tmp/dist-test-taskh2U6sE/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection URL: jdbc:derby:/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/metadb;create=true Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver Metastore connection User: APP Starting metastore schema initialization to 3.1.0 Initialization script hive-schema-3.1.0.derby.sql Initialization script completed schemaTool completed Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/derby.log 2020-01-07 02:52:51: Starting Hive Metastore Server Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/derby.log SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/tmp/dist-test-taskh2U6sE/thirdparty/src/hive-3.1.1/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/tmp/dist-test-taskh2U6sE/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Starting hive metastore on port 0 I0107 02:53:00.535708 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/data --block_manager=log --webserver_interface=localhost --ipki_ca_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:46043 --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --rpc_reuseport=true --master_addresses=127.2.165.190:46043,127.2.165.189:45229 --builtin_ntp_servers=127.2.165.148:37497 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin --hive_metastore_uris=thrift://127.0.0.1:37153 with env {} WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:53:00.603879 6302 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:53:00.604116 6302 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:53:00.605204 6302 flags.cc:404] Enabled experimental flag: --ipki_ca_key_size=1024 W0107 02:53:00.605231 6302 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:53:00.605237 6302 flags.cc:404] Enabled experimental flag: --tsk_num_rsa_bits=512 W0107 02:53:00.605247 6302 flags.cc:404] Enabled experimental flag: --rpc_reuseport=true I0107 02:53:00.607302 6302 master_main.cc:96] Master server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:37497 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/wal --hive_metastore_uris=thrift://127.0.0.1:37153 --ipki_ca_key_size=1024 --master_addresses=127.2.165.190:46043,127.2.165.189:45229 --ipki_server_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.190:46043 --rpc_reuseport=true --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/data/info.pb --webserver_interface=127.2.165.190 --webserver_port=0 --never_fsync=true --heap_profile_path=/tmp/kudu-master.6302 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/logs --logbuflevel=-1 --logtostderr=true Master server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 W0107 02:53:00.607543 6302 master_options.cc:58] Only 2 masters are specified by master_addresses_flag ('127.2.165.190:46043,127.2.165.189:45229'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters. I0107 02:53:00.617229 6302 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:53:00.620641 6302 hybrid_clock.cc:186] HybridClock initialized: now 1578365580620476 us; error 108 us; skew 500 ppm I0107 02:53:00.621488 6302 fs_manager.cc:263] Metadata directory not provided I0107 02:53:00.621536 6302 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:53:00.621827 6302 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:53:00.621852 6302 server_base.cc:453] Attempting to create new FS layout instead I0107 02:53:00.624369 6302 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/data/instance: uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" format_stamp: "Formatted at 2020-01-07 02:53:00 on dist-test-slave-dist-test-slave-3kpx" I0107 02:53:00.624680 6302 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/wal/instance: uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" format_stamp: "Formatted at 2020-01-07 02:53:00 on dist-test-slave-dist-test-slave-3kpx" I0107 02:53:00.626464 6302 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:53:00.633088 6302 fs_manager.cc:518] Time spent creating directory manager: real 0.008s user 0.002s sys 0.008s I0107 02:53:00.633832 6302 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:53:00.634347 6302 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:53:00.638514 6302 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s I0107 02:53:00.638651 6302 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/wal uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" format_stamp: "Formatted at 2020-01-07 02:53:00 on dist-test-slave-dist-test-slave-3kpx" I0107 02:53:00.638757 6302 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:53:00.666549 6302 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:53:00.673002 6302 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.190:46043 I0107 02:53:00.673964 6302 webserver.cc:382] Webserver started at http://127.2.165.190:42583/ using document root and password file I0107 02:53:00.674748 6302 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-0/data/info.pb I0107 02:53:00.678385 6363 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:53:00.682174 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master as pid 6302 I0107 02:53:00.682775 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/data --block_manager=log --webserver_interface=localhost --ipki_ca_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.189:45229 --webserver_interface=127.2.165.189 --webserver_port=0 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --rpc_reuseport=true --master_addresses=127.2.165.190:46043,127.2.165.189:45229 --builtin_ntp_servers=127.2.165.148:37497 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin --hive_metastore_uris=thrift://127.0.0.1:37153 with env {} I0107 02:53:00.682967 6363 sys_catalog.cc:306] member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } has no permanent_uuid. Determining permanent_uuid... I0107 02:53:00.694169 6363 sys_catalog.cc:306] member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } has no permanent_uuid. Determining permanent_uuid... W0107 02:53:00.694993 6363 consensus_peers.cc:623] Error getting permanent uuid from config peer 127.2.165.189:45229: Network error: Client connection negotiation failed: client connection to 127.2.165.189:45229: connect: Connection refused (error 111) I0107 02:53:00.736348 6363 consensus_peers.cc:633] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } attempt: 1 W0107 02:53:00.737519 6363 consensus_peers.cc:623] Error getting permanent uuid from config peer 127.2.165.189:45229: Network error: Client connection negotiation failed: client connection to 127.2.165.189:45229: connect: Connection refused (error 111) WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:53:00.760998 6364 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:53:00.761253 6364 flags.cc:404] Enabled unsafe flag: --never_fsync=true W0107 02:53:00.762538 6364 flags.cc:404] Enabled experimental flag: --ipki_ca_key_size=1024 W0107 02:53:00.762569 6364 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:53:00.762578 6364 flags.cc:404] Enabled experimental flag: --tsk_num_rsa_bits=512 W0107 02:53:00.762585 6364 flags.cc:404] Enabled experimental flag: --rpc_reuseport=true I0107 02:53:00.764686 6364 master_main.cc:96] Master server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:37497 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/wal --hive_metastore_uris=thrift://127.0.0.1:37153 --ipki_ca_key_size=1024 --master_addresses=127.2.165.190:46043,127.2.165.189:45229 --ipki_server_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.2.165.189:45229 --rpc_reuseport=true --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/data/info.pb --webserver_interface=127.2.165.189 --webserver_port=0 --never_fsync=true --heap_profile_path=/tmp/kudu-master.6364 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/logs --logbuflevel=-1 --logtostderr=true Master server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 W0107 02:53:00.764883 6364 master_options.cc:58] Only 2 masters are specified by master_addresses_flag ('127.2.165.190:46043,127.2.165.189:45229'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters. I0107 02:53:00.775255 6364 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:53:00.776502 6364 hybrid_clock.cc:186] HybridClock initialized: now 1578365580776410 us; error 75 us; skew 500 ppm I0107 02:53:00.777330 6364 fs_manager.cc:263] Metadata directory not provided I0107 02:53:00.777388 6364 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:53:00.777694 6364 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:53:00.777724 6364 server_base.cc:453] Attempting to create new FS layout instead I0107 02:53:00.780335 6364 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/data/instance: uuid: "bc75c8bfc57b4638a42370e77ac7a66a" format_stamp: "Formatted at 2020-01-07 02:53:00 on dist-test-slave-dist-test-slave-3kpx" I0107 02:53:00.780885 6364 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/wal/instance: uuid: "bc75c8bfc57b4638a42370e77ac7a66a" format_stamp: "Formatted at 2020-01-07 02:53:00 on dist-test-slave-dist-test-slave-3kpx" I0107 02:53:00.782858 6364 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:53:00.789407 6364 fs_manager.cc:518] Time spent creating directory manager: real 0.008s user 0.010s sys 0.000s I0107 02:53:00.790197 6364 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:53:00.790800 6364 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:53:00.795060 6364 fs_manager.cc:434] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s I0107 02:53:00.795190 6364 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/wal uuid: "bc75c8bfc57b4638a42370e77ac7a66a" format_stamp: "Formatted at 2020-01-07 02:53:00 on dist-test-slave-dist-test-slave-3kpx" I0107 02:53:00.795316 6364 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:53:00.800768 6363 consensus_peers.cc:633] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } attempt: 2 W0107 02:53:00.801813 6363 consensus_peers.cc:623] Error getting permanent uuid from config peer 127.2.165.189:45229: Network error: Client connection negotiation failed: client connection to 127.2.165.189:45229: connect: Connection refused (error 111) I0107 02:53:00.846400 6364 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:53:00.852845 6364 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.189:45229 I0107 02:53:00.853888 6364 webserver.cc:382] Webserver started at http://127.2.165.189:42679/ using document root and password file I0107 02:53:00.854831 6364 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/master-1/data/info.pb I0107 02:53:00.858510 6427 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:53:00.860239 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-master as pid 6364 I0107 02:53:00.860847 2710 external_mini_cluster.cc:1043] Running /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/wal --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/data --block_manager=log --rpc_bind_addresses=127.2.165.129:0 --local_ip_for_outbound_sockets=127.2.165.129 --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:46043,127.2.165.189:45229 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/logs --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --builtin_ntp_servers=127.2.165.148:37497 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin with env {} I0107 02:53:00.864071 6427 sys_catalog.cc:306] member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } has no permanent_uuid. Determining permanent_uuid... I0107 02:53:00.872005 6363 consensus_peers.cc:633] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } attempt: 3 I0107 02:53:00.873839 6427 sys_catalog.cc:306] member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } has no permanent_uuid. Determining permanent_uuid... I0107 02:53:00.880169 6427 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a: Bootstrap starting. I0107 02:53:00.880192 6363 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9: Bootstrap starting. I0107 02:53:00.914692 6363 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9: No blocks or log segments found. Creating new log. I0107 02:53:00.914744 6427 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a: No blocks or log segments found. Creating new log. I0107 02:53:00.916108 6363 log.cc:815] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9: Log is configured to *not* fsync() on all Append() calls I0107 02:53:00.916131 6427 log.cc:815] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a: Log is configured to *not* fsync() on all Append() calls I0107 02:53:00.919725 6427 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a: No bootstrap required, opened a new log I0107 02:53:00.919915 6363 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9: No bootstrap required, opened a new log I0107 02:53:00.923187 6427 raft_consensus.cc:340] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } I0107 02:53:00.923312 6427 raft_consensus.cc:366] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:53:00.923377 6427 raft_consensus.cc:705] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc75c8bfc57b4638a42370e77ac7a66a, State: Initialized, Role: FOLLOWER I0107 02:53:00.923388 6363 raft_consensus.cc:340] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } I0107 02:53:00.923506 6363 raft_consensus.cc:366] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:53:00.923566 6363 raft_consensus.cc:705] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6ac6b3f79ee45388a88376e12a0a7f9, State: Initialized, Role: FOLLOWER I0107 02:53:00.923952 6427 consensus_queue.cc:231] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } I0107 02:53:00.924008 6363 consensus_queue.cc:231] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } I0107 02:53:00.924342 6427 sys_catalog.cc:339] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:53:00.924374 6427 sys_catalog.cc:342] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [sys.catalog]: This master's current role is: FOLLOWER I0107 02:53:00.924384 6363 sys_catalog.cc:339] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:53:00.924408 6363 sys_catalog.cc:342] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [sys.catalog]: This master's current role is: FOLLOWER I0107 02:53:00.925316 6427 sys_catalog.cc:436] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [sys.catalog]: configured and running, proceeding with master startup. I0107 02:53:00.925325 6363 sys_catalog.cc:436] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [sys.catalog]: configured and running, proceeding with master startup. WARNING: Logging before InitGoogleLogging() is written to STDERR W0107 02:53:00.925333 6428 flags.cc:404] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0107 02:53:00.925588 6428 flags.cc:404] Enabled unsafe flag: --never_fsync=true I0107 02:53:00.926688 6434 sys_catalog.cc:339] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:53:00.926898 6435 sys_catalog.cc:339] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:53:00.926904 6434 sys_catalog.cc:342] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [sys.catalog]: This master's current role is: FOLLOWER I0107 02:53:00.926929 6435 sys_catalog.cc:342] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [sys.catalog]: This master's current role is: FOLLOWER I0107 02:53:00.927927 6440 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: bc75c8bfc57b4638a42370e77ac7a66a, Raft Consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:53:00.928320 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } W0107 02:53:00.933245 6443 catalog_manager.cc:1158] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9: acquiring CA information for follower catalog manager: Not found: root CA entry not found W0107 02:53:00.933297 6443 catalog_manager.cc:630] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry W0107 02:53:00.939966 6442 catalog_manager.cc:1158] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a: acquiring CA information for follower catalog manager: Not found: root CA entry not found W0107 02:53:00.940845 6428 flags.cc:404] Enabled experimental flag: --ipki_server_key_size=1024 W0107 02:53:00.940951 6428 flags.cc:404] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.165.129 W0107 02:53:00.946480 6442 catalog_manager.cc:630] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry I0107 02:53:00.947819 6428 tablet_server_main.cc:76] Tablet server non-default flags: --builtin_ntp_poll_interval_ms=100 --builtin_ntp_servers=127.2.165.148:37497 --ntp_initial_sync_wait_secs=10 --time_source=builtin --fs_data_dirs=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/data --fs_wal_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.2.165.129:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/data/info.pb --webserver_interface=127.2.165.129 --webserver_port=0 --tserver_master_addrs=127.2.165.190:46043,127.2.165.189:45229 --never_fsync=true --heap_profile_path=/tmp/kudu-tserver.6428 --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.2.165.129 --log_dir=/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.11.0.7.1.0.0-SNAPSHOT revision a4ff810dd64883133db852e9479c89ff2df803d7 build type DEBUG built by jenkins at 06 Jan 2020 18:27:29 PST on quasar-zycntq-1.vpc.cloudera.com build id 568631 I0107 02:53:00.992229 6428 hybrid_clock.cc:168] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize I0107 02:53:00.994508 6428 hybrid_clock.cc:186] HybridClock initialized: now 1578365580994425 us; error 63 us; skew 500 ppm I0107 02:53:00.995792 6428 fs_manager.cc:263] Metadata directory not provided I0107 02:53:00.995960 6428 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0107 02:53:00.996403 6428 server_base.cc:452] Could not load existing FS layout: Not found: could not find a healthy instance file I0107 02:53:00.996506 6428 server_base.cc:453] Attempting to create new FS layout instead I0107 02:53:01.008044 6428 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/data/instance: uuid: "cf494eb84b474273a0b20b0889bb6351" format_stamp: "Formatted at 2020-01-07 02:53:00 on dist-test-slave-dist-test-slave-3kpx" I0107 02:53:01.008435 6428 fs_manager.cc:617] Generated new instance metadata in path /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/wal/instance: uuid: "cf494eb84b474273a0b20b0889bb6351" format_stamp: "Formatted at 2020-01-07 02:53:00 on dist-test-slave-dist-test-slave-3kpx" I0107 02:53:01.010242 6428 dir_util.cc:225] Instance is unhealthy: Not found: Failed to read metadata file from /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/data/data/block_manager_instance: No such file or directory (error 2) I0107 02:53:01.034559 6428 fs_manager.cc:518] Time spent creating directory manager: real 0.026s user 0.011s sys 0.001s I0107 02:53:01.035414 6428 env_posix.cc:1680] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0107 02:53:01.036083 6428 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0107 02:53:01.040988 6428 fs_manager.cc:434] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0107 02:53:01.041154 6428 fs_manager.cc:451] Opened local filesystem: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/data,/tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/wal uuid: "cf494eb84b474273a0b20b0889bb6351" format_stamp: "Formatted at 2020-01-07 02:53:00 on dist-test-slave-dist-test-slave-3kpx" I0107 02:53:01.041287 6428 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/wal metadata directory: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/wal 1 data directories: /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0107 02:53:01.075175 6428 env_posix.cc:1680] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0107 02:53:01.077773 6428 ts_tablet_manager.cc:395] Loaded tablet metadata (0 total tablets, 0 live tablets) I0107 02:53:01.077811 6428 ts_tablet_manager.cc:415] Registered 0 tablets I0107 02:53:01.085582 6428 rpc_server.cc:211] RPC server started. Bound to: 127.2.165.129:42223 I0107 02:53:01.086714 6428 webserver.cc:382] Webserver started at http://127.2.165.129:33437/ using document root and password file I0107 02:53:01.087323 6428 server_base.cc:629] Dumped server information to /tmp/dist-test-taskh2U6sE/test-tmp/kudu-tool-test.2.ToolTest.TestHmsIgnoresDifferentMasters.1578365549852848-2710/minicluster-data/ts-0/data/info.pb I0107 02:53:01.088605 2710 external_mini_cluster.cc:1105] Started /tmp/dist-test-taskh2U6sE/build/debug/bin/kudu-tserver as pid 6428 W0107 02:53:01.098443 6383 tablet.cc:1792] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a: Can't schedule compaction. Clean time has not been advanced past its initial value. I0107 02:53:01.107123 6569 heartbeater.cc:322] Connected to a master server at 127.2.165.190:46043 I0107 02:53:01.107131 6568 heartbeater.cc:322] Connected to a master server at 127.2.165.189:45229 I0107 02:53:01.107329 6568 heartbeater.cc:406] Registering TS with master... I0107 02:53:01.107933 6568 heartbeater.cc:455] Master 127.2.165.189:45229 requested a full tablet report, sending... I0107 02:53:01.107329 6569 heartbeater.cc:406] Registering TS with master... I0107 02:53:01.108577 6569 heartbeater.cc:455] Master 127.2.165.190:46043 requested a full tablet report, sending... I0107 02:53:01.109769 6394 ts_manager.cc:192] Registered new tserver with Master: cf494eb84b474273a0b20b0889bb6351 (127.2.165.129:42223) I0107 02:53:01.109772 6330 ts_manager.cc:192] Registered new tserver with Master: cf494eb84b474273a0b20b0889bb6351 (127.2.165.129:42223) I0107 02:53:01.112264 2710 external_mini_cluster.cc:721] 1 TS(s) registered with all masters I0107 02:53:01.147550 6434 raft_consensus.cc:472] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0107 02:53:01.147796 6434 raft_consensus.cc:494] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } I0107 02:53:01.149266 6434 leader_election.cc:293] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6ac6b3f79ee45388a88376e12a0a7f9 (127.2.165.190:46043) I0107 02:53:01.150091 6340 tablet_service.cc:1307] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" is_pre_election: true I0107 02:53:01.150378 6340 raft_consensus.cc:2372] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc75c8bfc57b4638a42370e77ac7a66a in term 0. I0107 02:53:01.150969 6374 leader_election.cc:307] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: bc75c8bfc57b4638a42370e77ac7a66a, f6ac6b3f79ee45388a88376e12a0a7f9; no voters: I0107 02:53:01.151495 6434 raft_consensus.cc:2672] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [term 0 FOLLOWER]: Leader pre-election won for term 1 I0107 02:53:01.151564 6434 raft_consensus.cc:472] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0107 02:53:01.151585 6434 raft_consensus.cc:2918] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [term 0 FOLLOWER]: Advancing to term 1 I0107 02:53:01.158598 6434 raft_consensus.cc:494] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } I0107 02:53:01.159077 6434 leader_election.cc:293] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [CANDIDATE]: Term 1 election: Requested vote from peers f6ac6b3f79ee45388a88376e12a0a7f9 (127.2.165.190:46043) I0107 02:53:01.159677 6340 tablet_service.cc:1307] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" I0107 02:53:01.159781 6340 raft_consensus.cc:2918] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:53:01.166358 6340 raft_consensus.cc:2372] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc75c8bfc57b4638a42370e77ac7a66a in term 1. I0107 02:53:01.167016 6374 leader_election.cc:307] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: bc75c8bfc57b4638a42370e77ac7a66a, f6ac6b3f79ee45388a88376e12a0a7f9; no voters: I0107 02:53:01.167315 6434 raft_consensus.cc:2672] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:53:01.167547 6434 raft_consensus.cc:667] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [term 1 LEADER]: Becoming Leader. State: Replica: bc75c8bfc57b4638a42370e77ac7a66a, State: Running, Role: LEADER I0107 02:53:01.168099 6434 consensus_queue.cc:208] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } W0107 02:53:01.168105 6319 tablet.cc:1792] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9: Can't schedule compaction. Clean time has not been advanced past its initial value. I0107 02:53:01.169948 6434 sys_catalog.cc:339] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc75c8bfc57b4638a42370e77ac7a66a. Latest consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:53:01.169981 6434 sys_catalog.cc:342] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [sys.catalog]: This master's current role is: LEADER I0107 02:53:01.171360 6584 catalog_manager.cc:1089] Loading table and tablet metadata into memory... I0107 02:53:01.172693 6584 catalog_manager.cc:1100] Initializing Kudu internal certificate authority... I0107 02:53:01.329439 6340 raft_consensus.cc:1184] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [term 1 FOLLOWER]: Refusing update from remote peer bc75c8bfc57b4638a42370e77ac7a66a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch) I0107 02:53:01.330531 6582 consensus_queue.cc:984] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s I0107 02:53:01.335844 6435 sys_catalog.cc:339] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc75c8bfc57b4638a42370e77ac7a66a. Latest consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:53:01.335891 6435 sys_catalog.cc:342] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [sys.catalog]: This master's current role is: FOLLOWER I0107 02:53:01.338521 6434 sys_catalog.cc:339] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:53:01.338567 6434 sys_catalog.cc:342] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a [sys.catalog]: This master's current role is: LEADER I0107 02:53:01.339179 6435 sys_catalog.cc:339] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:53:01.339215 6435 sys_catalog.cc:342] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9 [sys.catalog]: This master's current role is: FOLLOWER I0107 02:53:01.339251 6586 mvcc.cc:203] Tried to move safe_time back from 6464985421115744256 to 6464985420466733056. Current Snapshot: MvccSnapshot[committed={T|T < 6464985421115744256}] I0107 02:53:01.340003 6584 catalog_manager.cc:984] Generated new certificate authority record I0107 02:53:01.341089 6584 catalog_manager.cc:1109] Loading token signing keys... I0107 02:53:01.349740 6584 catalog_manager.cc:4455] T 00000000000000000000000000000000 P bc75c8bfc57b4638a42370e77ac7a66a: Generated new TSK 0 I0107 02:53:01.350477 6584 catalog_manager.cc:1119] Initializing in-progress tserver states... I0107 02:53:01.351197 6584 catalog_manager.cc:1131] Loading latest processed Hive Metastore notification log event ID... I0107 02:53:01.370332 6394 catalog_manager.cc:1454] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47288: name: "default.table" schema { columns { name: "foo" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { } partition_schema { range_schema { columns { name: "foo" } } } I0107 02:53:01.866204 6503 tablet_service.cc:1034] Processing CreateTablet for tablet 1e2d03c3d5f24db880d35a0412af3626 (table=default.table [id=da09e09c49534423a92a077393e7bb78]), partition=RANGE (foo) PARTITION UNBOUNDED I0107 02:53:01.866890 6503 data_dirs.cc:394] Could only allocate 1 dirs of requested 3 for tablet 1e2d03c3d5f24db880d35a0412af3626. 1 dirs total, 0 dirs full, 0 dirs failed I0107 02:53:01.874788 6592 tablet_bootstrap.cc:439] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351: Bootstrap starting. I0107 02:53:01.876763 6592 tablet_bootstrap.cc:590] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351: No blocks or log segments found. Creating new log. I0107 02:53:01.877641 6592 log.cc:815] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351: Log is configured to *not* fsync() on all Append() calls I0107 02:53:01.878940 6592 tablet_bootstrap.cc:439] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351: No bootstrap required, opened a new log I0107 02:53:01.879134 6592 ts_tablet_manager.cc:1103] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s I0107 02:53:01.881634 6592 raft_consensus.cc:340] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf494eb84b474273a0b20b0889bb6351" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42223 } } I0107 02:53:01.881721 6592 raft_consensus.cc:366] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0107 02:53:01.881758 6592 raft_consensus.cc:705] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf494eb84b474273a0b20b0889bb6351, State: Initialized, Role: FOLLOWER I0107 02:53:01.882263 6592 consensus_queue.cc:231] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf494eb84b474273a0b20b0889bb6351" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42223 } } I0107 02:53:01.882334 6592 raft_consensus.cc:380] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0107 02:53:01.882452 6592 raft_consensus.cc:472] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0107 02:53:01.882589 6592 raft_consensus.cc:2918] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 [term 0 FOLLOWER]: Advancing to term 1 I0107 02:53:01.883512 6592 raft_consensus.cc:494] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf494eb84b474273a0b20b0889bb6351" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42223 } } I0107 02:53:01.883910 6592 leader_election.cc:307] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf494eb84b474273a0b20b0889bb6351; no voters: I0107 02:53:01.884227 6592 leader_election.cc:293] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 [CANDIDATE]: Term 1 election: Requested vote from peers I0107 02:53:01.885066 6592 ts_tablet_manager.cc:1131] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s I0107 02:53:01.886621 6594 raft_consensus.cc:2672] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 [term 1 FOLLOWER]: Leader election won for term 1 I0107 02:53:01.886885 6594 raft_consensus.cc:667] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 [term 1 LEADER]: Becoming Leader. State: Replica: cf494eb84b474273a0b20b0889bb6351, State: Running, Role: LEADER I0107 02:53:01.887221 6594 consensus_queue.cc:208] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf494eb84b474273a0b20b0889bb6351" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42223 } } I0107 02:53:01.888141 6394 master_service.cc:351] Signed X509 certificate for tserver {username='slave'} at 127.2.165.129:44831 I0107 02:53:01.890058 6568 heartbeater.cc:447] Master 127.2.165.189:45229 was elected leader, sending a full tablet report... I0107 02:53:01.891325 6394 catalog_manager.cc:4143] T 1e2d03c3d5f24db880d35a0412af3626 P cf494eb84b474273a0b20b0889bb6351 reported cstate change: term changed from 0 to 1, leader changed from to cf494eb84b474273a0b20b0889bb6351 (127.2.165.129). New cstate: current_term: 1 leader_uuid: "cf494eb84b474273a0b20b0889bb6351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf494eb84b474273a0b20b0889bb6351" member_type: VOTER last_known_addr { host: "127.2.165.129" port: 42223 } health_report { overall_health: HEALTHY } } } I0107 02:53:01.936691 6443 catalog_manager.cc:1156] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9: acquiring CA information for follower catalog manager: success I0107 02:53:01.937959 6443 catalog_manager.cc:1184] T 00000000000000000000000000000000 P f6ac6b3f79ee45388a88376e12a0a7f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0 I0107 02:53:15.928707 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:53:30.929154 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:53:45.929605 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:54:00.930042 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:54:15.930527 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:54:30.931027 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:54:45.931493 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:55:00.931927 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:55:01.038519 6383 maintenance_manager.cc:326] P bc75c8bfc57b4638a42370e77ac7a66a: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033365 I0107 02:55:01.058338 6382 maintenance_manager.cc:527] P bc75c8bfc57b4638a42370e77ac7a66a: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.016s sys 0.000s Metrics: {"bytes_written":6697,"cfile_init":1,"dirs.queue_time_us":245,"dirs.run_cpu_time_us":147,"dirs.run_wall_time_us":663,"drs_written":1,"lbm_read_time_us":52,"lbm_reads_lt_1ms":4,"lbm_write_time_us":334,"lbm_writes_lt_1ms":27,"rows_written":5,"thread_start_us":163,"threads_started":2,"wal-append.queue_time_us":160} I0107 02:55:01.059082 6383 maintenance_manager.cc:326] P bc75c8bfc57b4638a42370e77ac7a66a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 396 bytes on disk I0107 02:55:01.059856 6382 maintenance_manager.cc:527] P bc75c8bfc57b4638a42370e77ac7a66a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":56,"lbm_reads_lt_1ms":4} I0107 02:55:01.122133 6319 maintenance_manager.cc:326] P f6ac6b3f79ee45388a88376e12a0a7f9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033388 I0107 02:55:01.141829 6318 maintenance_manager.cc:527] P f6ac6b3f79ee45388a88376e12a0a7f9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.016s sys 0.000s Metrics: {"bytes_written":6697,"cfile_init":1,"dirs.queue_time_us":314,"dirs.run_cpu_time_us":169,"dirs.run_wall_time_us":705,"drs_written":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4,"lbm_write_time_us":396,"lbm_writes_lt_1ms":27,"rows_written":5,"thread_start_us":188,"threads_started":2,"wal-append.queue_time_us":165} I0107 02:55:01.142841 6319 maintenance_manager.cc:326] P f6ac6b3f79ee45388a88376e12a0a7f9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 396 bytes on disk I0107 02:55:01.143730 6318 maintenance_manager.cc:527] P f6ac6b3f79ee45388a88376e12a0a7f9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":65,"lbm_reads_lt_1ms":4} I0107 02:55:15.932410 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:55:30.932893 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:55:45.933356 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:56:00.933804 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:56:15.934235 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:56:30.934708 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:56:45.935155 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:57:00.935614 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:57:15.936102 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:57:30.936545 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:57:45.937013 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:58:00.937475 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:58:15.937924 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:58:30.938350 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:58:45.938967 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:59:00.939424 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:59:15.939849 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:59:30.940312 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 02:59:45.940755 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:00:00.941195 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:00:15.941654 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:00:30.942014 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:00:45.942448 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:01:00.942909 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:01:15.943392 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:01:30.943877 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:01:45.944375 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:02:00.944821 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:02:15.945258 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:02:30.945778 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:02:45.946228 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:03:00.946673 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:03:15.947141 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:03:30.947547 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:03:45.947957 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:04:00.948379 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:04:15.948772 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:04:30.949168 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:04:45.949594 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:05:00.950031 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:05:15.950492 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:05:30.950919 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:05:45.951383 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:06:00.951804 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:06:15.952224 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:06:30.952656 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } I0107 03:06:45.953116 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } ************************ BEGIN STACKS ************************** I0107 03:07:00.953589 6441 hms_notification_log_listener.cc:227] Skipping Hive Metastore notification log poll: Illegal state: Not the leader. Local UUID: f6ac6b3f79ee45388a88376e12a0a7f9, Raft Consensus state: current_term: 1 leader_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ac6b3f79ee45388a88376e12a0a7f9" member_type: VOTER last_known_addr { host: "127.2.165.190" port: 46043 } } peers { permanent_uuid: "bc75c8bfc57b4638a42370e77ac7a66a" member_type: VOTER last_known_addr { host: "127.2.165.189" port: 45229 } } } [New LWP 6579] [New LWP 6578] [New LWP 6577] [New LWP 6576] [New LWP 6043] [New LWP 2814] [New LWP 2713] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". 0x00007fc4a81f4693 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 81 ../sysdeps/unix/syscall-template.S: No such file or directory. Id Target Id Frame 8 Thread 0x7fc4a7698700 (LWP 2713) "kudu-tool-test" 0x00007fc4b1df3ed9 in __libc_waitpid (pid=7462, stat_loc=0x7fc4a769769c, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:40 7 Thread 0x7fc4a56a7700 (LWP 2814) "kernel-watcher-" pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 6 Thread 0x7fc460de4700 (LWP 6043) "rpc reactor-604" 0x00007fc4a81f4693 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 5 Thread 0x7fc4a4ea6700 (LWP 6576) "rpc reactor-657" 0x00007fc4a81f4693 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 4 Thread 0x7fc4a2667700 (LWP 6577) "rpc reactor-657" 0x00007fc4a81f4693 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 3 Thread 0x7fc4a3669700 (LWP 6578) "rpc reactor-657" 0x00007fc4a81f4693 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 2 Thread 0x7fc4a2e68700 (LWP 6579) "rpc reactor-657" 0x00007fc4a81f4693 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 * 1 Thread 0x7fc4bb46b8c0 (LWP 2710) "kudu-tool-test" 0x00007fc4a81f4693 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 Thread 8 (Thread 0x7fc4a7698700 (LWP 2713)): #0 0x00007fc4b1df3ed9 in __libc_waitpid (pid=7462, stat_loc=0x7fc4a769769c, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:40 #1 0x00007fc4ac4c2e42 in kudu::Subprocess::DoWait (this=0x7fc4a7697820, wait_status=0x0, mode=kudu::Subprocess::BLOCKING) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/subprocess.cc:799 #2 0x00007fc4ac4c0b07 in kudu::Subprocess::Wait (this=0x7fc4a7697820, wait_status=0x0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/subprocess.cc:549 #3 0x00007fc4ac42e991 in kudu::PstackWatcher::RunStackDump (argv=std::vector of length 12, capacity 16 = {...}) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/pstack_watcher.cc:233 #4 0x00007fc4ac42e546 in kudu::PstackWatcher::RunGdbStackDump (pid=2710, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/pstack_watcher.cc:210 #5 0x00007fc4ac42de28 in kudu::PstackWatcher::DumpPidStacks (pid=2710, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/pstack_watcher.cc:169 #6 0x00007fc4ac42ddc3 in kudu::PstackWatcher::DumpStacks (flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/pstack_watcher.cc:161 #7 0x00007fc4b97b29b2 in kudu::::operator()(void) const (__closure=0x2ad7198) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:60 #8 0x00007fc4b97b4cec in std::_Bind_simple()>::_M_invoke<>(std::_Index_tuple<>) (this=0x2ad7198) at /opt/rh/devtoolset-3/root/usr/include/c++/4.9.2/functional:1700 #9 0x00007fc4b97b4b7b in std::_Bind_simple()>::operator()(void) (this=0x2ad7198) at /opt/rh/devtoolset-3/root/usr/include/c++/4.9.2/functional:1688 #10 0x00007fc4b97b4a70 in std::thread::_Impl()> >::_M_run(void) (this=0x2ad7180) at /opt/rh/devtoolset-3/root/usr/include/c++/4.9.2/thread:115 #11 0x00007fc4a8a8ca60 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #12 0x00007fc4b1dec184 in start_thread (arg=0x7fc4a7698700) at pthread_create.c:312 #13 0x00007fc4a81f3ffd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 7 (Thread 0x7fc4a56a7700 (LWP 2814)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fc4ac3bcf3c in kudu::ConditionVariable::WaitUntil (this=0x2a480d8, until=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/condition_variable.cc:87 #2 0x00007fc4b87f5776 in kudu::CountDownLatch::WaitUntil (this=0x2a480a0, when=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/countdown_latch.h:84 #3 0x00007fc4b87f580b in kudu::CountDownLatch::WaitFor (this=0x2a480a0, delta=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/countdown_latch.h:94 #4 0x00007fc4ac43fc6b in kudu::KernelStackWatchdog::RunThread (this=0x2a48000) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/kernel_stack_watchdog.cc:134 #5 0x00007fc4ac444655 in boost::_mfi::mf0::operator() (this=0x2a79070, p=0x2a48000) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/mem_fn_template.hpp:49 #6 0x00007fc4ac44413a in boost::_bi::list1 >::operator(), boost::_bi::list0> (this=0x2a79080, f=..., a=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/bind.hpp:259 #7 0x00007fc4ac443c9f in boost::_bi::bind_t, boost::_bi::list1 > >::operator() (this=0x2a79070) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/bind.hpp:1222 #8 0x00007fc4ac443724 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke (function_obj_ptr=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/function/function_template.hpp:159 #9 0x00007fc4ae5464a6 in boost::function0::operator() (this=0x2a79068) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/function/function_template.hpp:771 #10 0x00007fc4ac4ceaa6 in kudu::Thread::SuperviseThread (arg=0x2a79040) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:675 #11 0x00007fc4b1dec184 in start_thread (arg=0x7fc4a56a7700) at pthread_create.c:312 #12 0x00007fc4a81f3ffd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 6 (Thread 0x7fc460de4700 (LWP 6043)): #0 0x00007fc4a81f4693 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fc4ab27bc67 in epoll_poll (loop=0x2bdf600, timeout=0.099463522082942291) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/libev-4.20/ev_epoll.c:153 #2 0x00007fc4ab27f300 in ev_run (loop=0x2bdf600, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/libev-4.20/ev.c:3521 #3 0x00007fc4ae554079 in ev::loop_ref::run (this=0x2c3b8a8, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/ev++.h:211 #4 0x00007fc4ae550330 in kudu::rpc::ReactorThread::RunThread (this=0x2c3b8a0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/reactor.cc:497 #5 0x00007fc4ae55bcbb in boost::_mfi::mf0::operator() (this=0x2d6fe40, p=0x2c3b8a0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/mem_fn_template.hpp:49 #6 0x00007fc4ae55b610 in boost::_bi::list1 >::operator(), boost::_bi::list0> (this=0x2d6fe50, f=..., a=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/bind.hpp:259 #7 0x00007fc4ae55ae31 in boost::_bi::bind_t, boost::_bi::list1 > >::operator() (this=0x2d6fe40) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/bind.hpp:1222 #8 0x00007fc4ae55a2f8 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke (function_obj_ptr=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/function/function_template.hpp:159 #9 0x00007fc4ae5464a6 in boost::function0::operator() (this=0x2d6fe38) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/function/function_template.hpp:771 #10 0x00007fc4ac4ceaa6 in kudu::Thread::SuperviseThread (arg=0x2d6fe10) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:675 #11 0x00007fc4b1dec184 in start_thread (arg=0x7fc460de4700) at pthread_create.c:312 #12 0x00007fc4a81f3ffd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 5 (Thread 0x7fc4a4ea6700 (LWP 6576)): #0 0x00007fc4a81f4693 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fc4ab27bc67 in epoll_poll (loop=0x2bdc900, timeout=0.099587213079075809) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/libev-4.20/ev_epoll.c:153 #2 0x00007fc4ab27f300 in ev_run (loop=0x2bdc900, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/libev-4.20/ev.c:3521 #3 0x00007fc4ae554079 in ev::loop_ref::run (this=0x2c3aa08, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/ev++.h:211 #4 0x00007fc4ae550330 in kudu::rpc::ReactorThread::RunThread (this=0x2c3aa00) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/reactor.cc:497 #5 0x00007fc4ae55bcbb in boost::_mfi::mf0::operator() (this=0x2d6e030, p=0x2c3aa00) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/mem_fn_template.hpp:49 #6 0x00007fc4ae55b610 in boost::_bi::list1 >::operator(), boost::_bi::list0> (this=0x2d6e040, f=..., a=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/bind.hpp:259 #7 0x00007fc4ae55ae31 in boost::_bi::bind_t, boost::_bi::list1 > >::operator() (this=0x2d6e030) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/bind.hpp:1222 #8 0x00007fc4ae55a2f8 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke (function_obj_ptr=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/function/function_template.hpp:159 #9 0x00007fc4ae5464a6 in boost::function0::operator() (this=0x2d6e028) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/function/function_template.hpp:771 #10 0x00007fc4ac4ceaa6 in kudu::Thread::SuperviseThread (arg=0x2d6e000) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:675 #11 0x00007fc4b1dec184 in start_thread (arg=0x7fc4a4ea6700) at pthread_create.c:312 #12 0x00007fc4a81f3ffd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 4 (Thread 0x7fc4a2667700 (LWP 6577)): #0 0x00007fc4a81f4693 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fc4ab27bc67 in epoll_poll (loop=0x2bdfa80, timeout=0.099641651079082294) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/libev-4.20/ev_epoll.c:153 #2 0x00007fc4ab27f300 in ev_run (loop=0x2bdfa80, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/libev-4.20/ev.c:3521 #3 0x00007fc4ae554079 in ev::loop_ref::run (this=0x301d708, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/ev++.h:211 #4 0x00007fc4ae550330 in kudu::rpc::ReactorThread::RunThread (this=0x301d700) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/reactor.cc:497 #5 0x00007fc4ae55bcbb in boost::_mfi::mf0::operator() (this=0x2c65f10, p=0x301d700) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/mem_fn_template.hpp:49 #6 0x00007fc4ae55b610 in boost::_bi::list1 >::operator(), boost::_bi::list0> (this=0x2c65f20, f=..., a=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/bind.hpp:259 #7 0x00007fc4ae55ae31 in boost::_bi::bind_t, boost::_bi::list1 > >::operator() (this=0x2c65f10) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/bind.hpp:1222 #8 0x00007fc4ae55a2f8 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke (function_obj_ptr=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/function/function_template.hpp:159 #9 0x00007fc4ae5464a6 in boost::function0::operator() (this=0x2c65f08) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/function/function_template.hpp:771 #10 0x00007fc4ac4ceaa6 in kudu::Thread::SuperviseThread (arg=0x2c65ee0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:675 #11 0x00007fc4b1dec184 in start_thread (arg=0x7fc4a2667700) at pthread_create.c:312 #12 0x00007fc4a81f3ffd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 3 (Thread 0x7fc4a3669700 (LWP 6578)): #0 0x00007fc4a81f4693 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fc4ab27bc67 in epoll_poll (loop=0x2bded00, timeout=0.099655465079194983) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/libev-4.20/ev_epoll.c:153 #2 0x00007fc4ab27f300 in ev_run (loop=0x2bded00, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/libev-4.20/ev.c:3521 #3 0x00007fc4ae554079 in ev::loop_ref::run (this=0x301c388, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/ev++.h:211 #4 0x00007fc4ae550330 in kudu::rpc::ReactorThread::RunThread (this=0x301c380) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/reactor.cc:497 #5 0x00007fc4ae55bcbb in boost::_mfi::mf0::operator() (this=0x2c65d70, p=0x301c380) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/mem_fn_template.hpp:49 #6 0x00007fc4ae55b610 in boost::_bi::list1 >::operator(), boost::_bi::list0> (this=0x2c65d80, f=..., a=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/bind.hpp:259 #7 0x00007fc4ae55ae31 in boost::_bi::bind_t, boost::_bi::list1 > >::operator() (this=0x2c65d70) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/bind.hpp:1222 #8 0x00007fc4ae55a2f8 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke (function_obj_ptr=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/function/function_template.hpp:159 #9 0x00007fc4ae5464a6 in boost::function0::operator() (this=0x2c65d68) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/function/function_template.hpp:771 #10 0x00007fc4ac4ceaa6 in kudu::Thread::SuperviseThread (arg=0x2c65d40) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:675 #11 0x00007fc4b1dec184 in start_thread (arg=0x7fc4a3669700) at pthread_create.c:312 #12 0x00007fc4a81f3ffd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 2 (Thread 0x7fc4a2e68700 (LWP 6579)): #0 0x00007fc4a81f4693 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fc4ab27bc67 in epoll_poll (loop=0x2bdf180, timeout=0.099667029079228087) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/libev-4.20/ev_epoll.c:153 #2 0x00007fc4ab27f300 in ev_run (loop=0x2bdf180, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/libev-4.20/ev.c:3521 #3 0x00007fc4ae554079 in ev::loop_ref::run (this=0x301c868, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/ev++.h:211 #4 0x00007fc4ae550330 in kudu::rpc::ReactorThread::RunThread (this=0x301c860) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/reactor.cc:497 #5 0x00007fc4ae55bcbb in boost::_mfi::mf0::operator() (this=0x2c65ca0, p=0x301c860) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/mem_fn_template.hpp:49 #6 0x00007fc4ae55b610 in boost::_bi::list1 >::operator(), boost::_bi::list0> (this=0x2c65cb0, f=..., a=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/bind.hpp:259 #7 0x00007fc4ae55ae31 in boost::_bi::bind_t, boost::_bi::list1 > >::operator() (this=0x2c65ca0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/bind.hpp:1222 #8 0x00007fc4ae55a2f8 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke (function_obj_ptr=...) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/function/function_template.hpp:159 #9 0x00007fc4ae5464a6 in boost::function0::operator() (this=0x2c65c98) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/function/function_template.hpp:771 #10 0x00007fc4ac4ceaa6 in kudu::Thread::SuperviseThread (arg=0x2c65c70) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:675 #11 0x00007fc4b1dec184 in start_thread (arg=0x7fc4a2e68700) at pthread_create.c:312 #12 0x00007fc4a81f3ffd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 1 (Thread 0x7fc4bb46b8c0 (LWP 2710)): #0 0x00007fc4a81f4693 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fc4ab27bc67 in epoll_poll (loop=0x2bddf80, timeout=59.743000000000002) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/libev-4.20/ev_epoll.c:153 #2 0x00007fc4ab27f300 in ev_run (loop=0x2bddf80, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/libev-4.20/ev.c:3521 #3 0x00007fc4ae554079 in ev::loop_ref::run (this=0x7fff69e5e760, flags=0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/ev++.h:211 #4 0x00007fc4ac4bee51 in kudu::(anonymous namespace)::ReadFdsFully (progname="/tmp/dist-test-taskh2U6sE/build/debug/bin/kudu", fds=std::vector of length 2, capacity 2 = {...}, out=0x7fff69e5e890) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/subprocess.cc:243 #5 0x00007fc4ac4c269b in kudu::Subprocess::Call (argv=std::vector of length 7, capacity 8 = {...}, stdin_in="", stdout_out=0x7fff69e5ebe0, stderr_out=0x7fff69e5ebd0, env_vars=std::map with 0 elements) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/subprocess.cc:755 #6 0x00007fc4ba78a649 in kudu::tools::RunKuduTool (args=std::vector of length 3, capacity 3 = {...}, out=0x7fff69e5ebe0, err=0x7fff69e5ebd0, in="") at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_test_util.cc:64 #7 0x00000000005e3e1e in kudu::tools::ToolTest::RunTool (this=0x2b0a400, arg_str="hms check 127.2.165.190:46043,127.2.165.189:45229", stdout=0x7fff69e5f4b0, stderr=0x7fff69e5f440, stdout_lines=0x0, stderr_lines=0x0, in="") at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/kudu-tool-test.cc:247 #8 0x00000000005e4c11 in kudu::tools::ToolTest::RunActionStdoutStderrString (this=0x2b0a400, arg_str="hms check 127.2.165.190:46043,127.2.165.189:45229", stdout=0x7fff69e5f4b0, stderr=0x7fff69e5f440) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/kudu-tool-test.cc:304 #9 0x00000000005a567b in kudu::tools::ToolTest_TestHmsIgnoresDifferentMasters_Test::TestBody (this=0x2b0a400) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/kudu-tool-test.cc:4301 #10 0x00007fc4b8adab98 in HandleSehExceptionsInMethodIfSupported (location=0x7fc4b8aed098 "the test body", method=, object=) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402 #11 testing::internal::HandleExceptionsInMethodIfSupported (object=object@entry=0x2b0a400, method=(void (testing::Test::*)(testing::Test * const)) 0x5a43c6 , location=location@entry=0x7fc4b8aed098 "the test body") at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 #12 0x00007fc4b8ac81b2 in testing::Test::Run (this=0x2b0a400) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2475 #13 0x00007fc4b8ac82f8 in testing::TestInfo::Run (this=0x2aca840) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2656 #14 0x00007fc4b8ac83d5 in testing::TestCase::Run (this=0x2a360e0) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2774 #15 0x00007fc4b8aceed8 in testing::internal::UnitTestImpl::RunAllTests (this=0x2ac4000) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4649 #16 0x00007fc4b8adb0a8 in HandleSehExceptionsInMethodIfSupported (location=0x7fc4b8aee560 "auxiliary test code (environments or event listeners)", method=, object=) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402 #17 testing::internal::HandleExceptionsInMethodIfSupported (object=0x2ac4000, method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x7fc4b8acec70 , location=location@entry=0x7fc4b8aee560 "auxiliary test code (environments or event listeners)") at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 #18 0x00007fc4b8ac84ad in testing::UnitTest::Run (this=0x7fc4b8d02300 ) at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4260 #19 0x00007fc4b97b4f6f in RUN_ALL_TESTS () at /data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/gtest/gtest.h:2233 #20 0x00007fc4b97b2d80 in main (argc=1, argv=0x7fff69e605e8) at /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:106 #21 0x00007fc4a8117f45 in __libc_start_main (main=0x52deb0 , argc=3, argv=0x7fff69e605d8, init=, fini=, rtld_fini=, stack_end=0x7fff69e605c8) at libc-start.c:287 #22 0x0000000000530089 in _start () ************************* END STACKS *************************** F0107 03:07:12.083597 2713 test_main.cc:63] Maximum unit test time exceeded (870 sec) *** Check failure stack trace: *** *** Aborted at 1578366432 (unix time) try "date -d @1578366432" if you are using GNU date *** PC: @ 0x7fc4a812cc37 gsignal *** SIGABRT (@0x3e800000a96) received by PID 2710 (TID 0x7fc4a7698700) from PID 2710; stack trace: *** @ 0x7fc4b1df4330 (unknown) at ??:0 @ 0x7fc4a812cc37 gsignal at ??:0 @ 0x7fc4a8130028 abort at ??:0 @ 0x7fc4aa4bda29 google::logging_fail() at ??:0 @ 0x7fc4aa4bf31d google::LogMessage::Fail() at ??:0 @ 0x7fc4aa4c11dd google::LogMessage::SendToLog() at ??:0 @ 0x7fc4aa4bee59 google::LogMessage::Flush() at ??:0 @ 0x7fc4aa4c1c7f google::LogMessageFatal::~LogMessageFatal() at ??:0 @ 0x7fc4b97b2acb _ZZN4kuduL27CreateAndStartTimeoutThreadEvENKUlvE_clEv at ??:0 @ 0x7fc4b97b4cec _ZNSt12_Bind_simpleIFZN4kuduL27CreateAndStartTimeoutThreadEvEUlvE_vEE9_M_invokeIIEEEvSt12_Index_tupleIIXspT_EEE at ??:0 @ 0x7fc4b97b4b7b _ZNSt12_Bind_simpleIFZN4kuduL27CreateAndStartTimeoutThreadEvEUlvE_vEEclEv at ??:0 @ 0x7fc4b97b4a70 _ZNSt6thread5_ImplISt12_Bind_simpleIFZN4kuduL27CreateAndStartTimeoutThreadEvEUlvE_vEEE6_M_runEv at ??:0 @ 0x7fc4a8a8ca60 (unknown) at ??:0 @ 0x7fc4b1dec184 start_thread at ??:0 @ 0x7fc4a81f3ffd clone at ??:0