20:42:07.128 [INFO - main] (BaseKuduTest.java:77) Setting up before class... 20:42:07.142 [INFO - main] (MiniKuduCluster.java:115) Starting 1 masters... 20:42:07.142 [INFO - main] (MiniKuduCluster.java:157) Starting 1 masters... 20:42:07.159 [INFO - main] (MiniKuduCluster.java:224) Starting process: /home/jenkins-slave/workspace/kudu-1/build/latest/bin/kudu-master --flagfile=/home/jenkins-slave/workspace/kudu-1/java/kudu-client/target/test-classes/flags --fs_wal_dir=/home/jenkins-slave/workspace/kudu-1/build/test-tmp/master-0-1466714527156 --fs_data_dirs=/home/jenkins-slave/workspace/kudu-1/build/test-tmp/master-0-1466714527156 --webserver_interface=127.126.201.1 --local_ip_for_outbound_sockets=127.126.201.1 --rpc_bind_addresses=127.126.201.1:64030 --webserver_port=64031 20:42:07.520 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.520512 32502 mem_tracker.cc:175] Creating tracker root 20:42:07.521 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.520814 32502 mem_tracker.cc:138] MemTracker: hard memory limit is 94.627029 GB 20:42:07.521 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.520872 32502 mem_tracker.cc:140] MemTracker: soft memory limit is 56.776218 GB 20:42:07.529 [INFO - main] (MiniKuduCluster.java:117) Starting 3 tablet servers... 20:42:07.530 [INFO - main] (MiniKuduCluster.java:224) Starting process: /home/jenkins-slave/workspace/kudu-1/build/latest/bin/kudu-tserver --flagfile=/home/jenkins-slave/workspace/kudu-1/java/kudu-client/target/test-classes/flags --fs_wal_dir=/home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-0-1466714527141 --fs_data_dirs=/home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-0-1466714527141 --flush_threshold_mb=1 --tserver_master_addrs=127.126.201.1:64030 --webserver_interface=127.126.201.1 --local_ip_for_outbound_sockets=127.126.201.1 --webserver_port=64033 --rpc_bind_addresses=127.126.201.1:64032 20:42:07.549 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.549242 32502 mem_tracker.cc:175] Creating tracker server->root 20:42:07.551 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.550933 32502 mem_tracker.cc:175] Creating tracker code_cache-sharded_lru_cache->root 20:42:07.609 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.609518 32502 master_main.cc:58] Initializing master server... 20:42:07.609 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.609742 32502 mem_tracker.cc:175] Creating tracker block_cache-sharded_lru_cache->root 20:42:07.610 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.610606 32502 hybrid_clock.cc:178] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 813729 20:42:07.611 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.611681 32502 fs_manager.cc:198] WAL root: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/master-0-1466714527156 20:42:07.611 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.611732 32502 fs_manager.cc:199] Metadata root: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/master-0-1466714527156 20:42:07.614 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.611757 32502 fs_manager.cc:200] Data roots: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/master-0-1466714527156 20:42:07.614 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.611774 32502 fs_manager.cc:201] All roots: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/master-0-1466714527156 20:42:07.614 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.611937 32502 mem_tracker.cc:175] Creating tracker log_block_manager->server->root 20:42:07.614 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.613554 32502 server_base.cc:164] Could not load existing FS layout: Not found: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/master-0-1466714527156/instance: No such file or directory (error 2) 20:42:07.614 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.613643 32502 server_base.cc:165] Creating new FS layout 20:42:07.617 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.616845 32502 fs_manager.cc:345] Generated new instance metadata in path /home/jenkins-slave/workspace/kudu-1/build/test-tmp/master-0-1466714527156/instance: 20:42:07.617 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) uuid: "8c2111a50593495bac4f03a5a25f85c3" 20:42:07.617 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) format_stamp: "Formatted at 2016-06-23 20:42:07 on jenkins-slave-c3oe" 20:42:07.618 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.618424 32502 pb_util.cc:258] Reading PB with version 2 starting at offset 16 20:42:07.618 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.618679 32502 pb_util.cc:258] Reading PB with version 2 starting at offset 555 20:42:07.661 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.661787 32510 pb_util.cc:258] Reading PB with version 2 starting at offset 16 20:42:07.662 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.661994 32510 pb_util.cc:258] Reading PB with version 2 starting at offset 559 20:42:07.662 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.662606 32502 fs_manager.cc:242] Opened local filesystem: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/master-0-1466714527156 20:42:07.662 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) uuid: "8c2111a50593495bac4f03a5a25f85c3" 20:42:07.663 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) format_stamp: "Formatted at 2016-06-23 20:42:07 on jenkins-slave-c3oe" 20:42:07.830 [INFO - kudu-master:64030] (MiniKuduCluster.java:405) I0623 20:42:07.830474 32502 master_main.cc:61] Starting Master server... 20:42:07.873 [INFO - main] (MiniKuduCluster.java:224) Starting process: /home/jenkins-slave/workspace/kudu-1/build/latest/bin/kudu-tserver --flagfile=/home/jenkins-slave/workspace/kudu-1/java/kudu-client/target/test-classes/flags --fs_wal_dir=/home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-1-1466714527141 --fs_data_dirs=/home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-1-1466714527141 --flush_threshold_mb=1 --tserver_master_addrs=127.126.201.1:64030 --webserver_interface=127.126.201.1 --local_ip_for_outbound_sockets=127.126.201.1 --webserver_port=64035 --rpc_bind_addresses=127.126.201.1:64034 20:42:07.880 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.880635 32505 mem_tracker.cc:175] Creating tracker root 20:42:07.881 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.880928 32505 mem_tracker.cc:138] MemTracker: hard memory limit is 94.627029 GB 20:42:07.881 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.880966 32505 mem_tracker.cc:140] MemTracker: soft memory limit is 56.776218 GB 20:42:07.881 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.881245 32505 mem_tracker.cc:175] Creating tracker server->root 20:42:07.882 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.882725 32505 mem_tracker.cc:175] Creating tracker code_cache-sharded_lru_cache->root 20:42:07.926 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.926802 32505 tablet_server_main.cc:54] Initializing tablet server... 20:42:07.927 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.927014 32505 mem_tracker.cc:175] Creating tracker block_cache-sharded_lru_cache->root 20:42:07.928 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.928215 32505 hybrid_clock.cc:178] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 813729 20:42:07.928 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.928789 32505 fs_manager.cc:198] WAL root: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-0-1466714527141 20:42:07.929 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.928827 32505 fs_manager.cc:199] Metadata root: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-0-1466714527141 20:42:07.929 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.928855 32505 fs_manager.cc:200] Data roots: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-0-1466714527141 20:42:07.929 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.928887 32505 fs_manager.cc:201] All roots: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-0-1466714527141 20:42:07.929 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.929131 32505 mem_tracker.cc:175] Creating tracker log_block_manager->server->root 20:42:07.931 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.930938 32505 server_base.cc:164] Could not load existing FS layout: Not found: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-0-1466714527141/instance: No such file or directory (error 2) 20:42:07.931 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.931031 32505 server_base.cc:165] Creating new FS layout 20:42:07.934 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.934427 32505 fs_manager.cc:345] Generated new instance metadata in path /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-0-1466714527141/instance: 20:42:07.934 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) uuid: "a3a75a7c31534c418f43d101d18101cb" 20:42:07.934 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) format_stamp: "Formatted at 2016-06-23 20:42:07 on jenkins-slave-c3oe" 20:42:07.936 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.936013 32505 pb_util.cc:258] Reading PB with version 2 starting at offset 16 20:42:07.936 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.936254 32505 pb_util.cc:258] Reading PB with version 2 starting at offset 555 20:42:07.937 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.937541 32525 pb_util.cc:258] Reading PB with version 2 starting at offset 16 20:42:07.937 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.937706 32525 pb_util.cc:258] Reading PB with version 2 starting at offset 559 20:42:07.938 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.938241 32505 fs_manager.cc:242] Opened local filesystem: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-0-1466714527141 20:42:07.938 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) uuid: "a3a75a7c31534c418f43d101d18101cb" 20:42:07.938 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) format_stamp: "Formatted at 2016-06-23 20:42:07 on jenkins-slave-c3oe" 20:42:07.943 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.943120 32505 fs_manager.cc:401] Ignoring hidden file in tablet metadata dir: . 20:42:07.943 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.943164 32505 fs_manager.cc:401] Ignoring hidden file in tablet metadata dir: .. 20:42:07.943 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.943562 32505 tablet_server_main.cc:57] Starting tablet server... 20:42:07.972 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.972642 32505 rpc_server.cc:164] RPC server started. Bound to: 127.126.201.1:64032 20:42:07.973 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.973115 32505 webserver.cc:124] Starting webserver on 127.126.201.1:64033 20:42:07.973 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.973155 32505 webserver.cc:135] Document root disabled 20:42:07.973 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.973779 32505 webserver.cc:218] Webserver started. Bound to: http://127.126.201.1:64033/ 20:42:07.974 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.974104 32635 heartbeater.cc:365] Heartbeat thread starting 20:42:07.974 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.974215 32635 heartbeater.cc:310] No valid master proxy. Connecting... 20:42:07.975 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.975733 32505 tablet_server_main.cc:60] Tablet server successfully started. 20:42:07.977 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.977568 32527 connection.cc:257] Transfer of RPC call RPC call kudu.master.MasterService.Ping -> {remote=127.126.201.1:64030, user_credentials={real_user=jenkins-slave, eff_user=}} aborted: Runtime error: RPC transfer destroyed before it finished sending 20:42:07.977 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) W0623 20:42:07.977704 32635 heartbeater.cc:402] Failed to heartbeat to 127.126.201.1:64030: Network error: Failed to ping master at 127.126.201.1:64030: Client connection negotiation failed: client connection to 127.126.201.1:64030: connect: Connection refused (error 111) 20:42:07.978 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.977782 32635 heartbeater.cc:310] No valid master proxy. Connecting... 20:42:07.978 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.978431 32527 connection.cc:257] Transfer of RPC call RPC call kudu.master.MasterService.Ping -> {remote=127.126.201.1:64030, user_credentials={real_user=jenkins-slave, eff_user=}} aborted: Runtime error: RPC transfer destroyed before it finished sending 20:42:07.978 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) W0623 20:42:07.978502 32635 heartbeater.cc:402] Failed to heartbeat to 127.126.201.1:64030: Network error: Failed to ping master at 127.126.201.1:64030: Client connection negotiation failed: client connection to 127.126.201.1:64030: connect: Connection refused (error 111) 20:42:07.978 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.978560 32635 heartbeater.cc:310] No valid master proxy. Connecting... 20:42:07.979 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.979234 32527 connection.cc:257] Transfer of RPC call RPC call kudu.master.MasterService.Ping -> {remote=127.126.201.1:64030, user_credentials={real_user=jenkins-slave, eff_user=}} aborted: Runtime error: RPC transfer destroyed before it finished sending 20:42:07.979 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) W0623 20:42:07.979308 32635 heartbeater.cc:402] Failed to heartbeat to 127.126.201.1:64030: Network error: Failed to ping master at 127.126.201.1:64030: Client connection negotiation failed: client connection to 127.126.201.1:64030: connect: Connection refused (error 111) 20:42:07.979 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) W0623 20:42:07.979339 32635 heartbeater.cc:283] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts. 20:42:07.979 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.979382 32635 heartbeater.cc:310] No valid master proxy. Connecting... 20:42:07.980 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) I0623 20:42:07.979930 32527 connection.cc:257] Transfer of RPC call RPC call kudu.master.MasterService.Ping -> {remote=127.126.201.1:64030, user_credentials={real_user=jenkins-slave, eff_user=}} aborted: Runtime error: RPC transfer destroyed before it finished sending 20:42:07.980 [INFO - kudu-tserver:64032] (MiniKuduCluster.java:405) W0623 20:42:07.980017 32635 heartbeater.cc:402] Failed to heartbeat to 127.126.201.1:64030: Network error: Failed to ping master at 127.126.201.1:64030: Client connection negotiation failed: client connection to 127.126.201.1:64030: connect: Connection refused (error 111) 20:42:08.173 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.173251 32518 mem_tracker.cc:175] Creating tracker root 20:42:08.173 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.173516 32518 mem_tracker.cc:138] MemTracker: hard memory limit is 94.627029 GB 20:42:08.173 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.173563 32518 mem_tracker.cc:140] MemTracker: soft memory limit is 56.776218 GB 20:42:08.174 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.173796 32518 mem_tracker.cc:175] Creating tracker server->root 20:42:08.175 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.175205 32518 mem_tracker.cc:175] Creating tracker code_cache-sharded_lru_cache->root 20:42:08.201 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.201782 32518 tablet_server_main.cc:54] Initializing tablet server... 20:42:08.201 [INFO - main] (MiniKuduCluster.java:224) Starting process: /home/jenkins-slave/workspace/kudu-1/build/latest/bin/kudu-tserver --flagfile=/home/jenkins-slave/workspace/kudu-1/java/kudu-client/target/test-classes/flags --fs_wal_dir=/home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-2-1466714527141 --fs_data_dirs=/home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-2-1466714527141 --flush_threshold_mb=1 --tserver_master_addrs=127.126.201.1:64030 --webserver_interface=127.126.201.1 --local_ip_for_outbound_sockets=127.126.201.1 --webserver_port=64037 --rpc_bind_addresses=127.126.201.1:64036 20:42:08.202 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.202013 32518 mem_tracker.cc:175] Creating tracker block_cache-sharded_lru_cache->root 20:42:08.203 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.203171 32518 hybrid_clock.cc:178] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 814229 20:42:08.203 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.203757 32518 fs_manager.cc:198] WAL root: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-1-1466714527141 20:42:08.203 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.203804 32518 fs_manager.cc:199] Metadata root: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-1-1466714527141 20:42:08.204 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.203824 32518 fs_manager.cc:200] Data roots: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-1-1466714527141 20:42:08.204 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.203840 32518 fs_manager.cc:201] All roots: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-1-1466714527141 20:42:08.204 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.204041 32518 mem_tracker.cc:175] Creating tracker log_block_manager->server->root 20:42:08.205 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.205869 32518 server_base.cc:164] Could not load existing FS layout: Not found: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-1-1466714527141/instance: No such file or directory (error 2) 20:42:08.206 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.205950 32518 server_base.cc:165] Creating new FS layout 20:42:08.209 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.209183 32518 fs_manager.cc:345] Generated new instance metadata in path /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-1-1466714527141/instance: 20:42:08.209 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) uuid: "1598153c6a454d70a5ea1dff26b06040" 20:42:08.209 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) format_stamp: "Formatted at 2016-06-23 20:42:08 on jenkins-slave-c3oe" 20:42:08.210 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.210830 32518 pb_util.cc:258] Reading PB with version 2 starting at offset 16 20:42:08.211 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.211052 32518 pb_util.cc:258] Reading PB with version 2 starting at offset 555 20:42:08.212 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.212404 32649 pb_util.cc:258] Reading PB with version 2 starting at offset 16 20:42:08.212 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.212672 32649 pb_util.cc:258] Reading PB with version 2 starting at offset 559 20:42:08.213 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.213466 32518 fs_manager.cc:242] Opened local filesystem: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-1-1466714527141 20:42:08.213 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) uuid: "1598153c6a454d70a5ea1dff26b06040" 20:42:08.213 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) format_stamp: "Formatted at 2016-06-23 20:42:08 on jenkins-slave-c3oe" 20:42:08.219 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.219076 32518 fs_manager.cc:401] Ignoring hidden file in tablet metadata dir: . 20:42:08.219 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.219131 32518 fs_manager.cc:401] Ignoring hidden file in tablet metadata dir: .. 20:42:08.219 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.219625 32518 tablet_server_main.cc:57] Starting tablet server... 20:42:08.252 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.252616 32518 rpc_server.cc:164] RPC server started. Bound to: 127.126.201.1:64034 20:42:08.253 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.253150 32518 webserver.cc:124] Starting webserver on 127.126.201.1:64035 20:42:08.253 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.253196 32518 webserver.cc:135] Document root disabled 20:42:08.254 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.254036 32518 webserver.cc:218] Webserver started. Bound to: http://127.126.201.1:64035/ 20:42:08.254 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.254428 32759 heartbeater.cc:365] Heartbeat thread starting 20:42:08.254 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.254560 32759 heartbeater.cc:310] No valid master proxy. Connecting... 20:42:08.255 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.255005 32518 tablet_server_main.cc:60] Tablet server successfully started. 20:42:08.270 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.270107 32651 connection.cc:257] Transfer of RPC call RPC call kudu.master.MasterService.Ping -> {remote=127.126.201.1:64030, user_credentials={real_user=jenkins-slave, eff_user=}} aborted: Runtime error: RPC transfer destroyed before it finished sending 20:42:08.270 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) W0623 20:42:08.270278 32759 heartbeater.cc:402] Failed to heartbeat to 127.126.201.1:64030: Network error: Failed to ping master at 127.126.201.1:64030: Client connection negotiation failed: client connection to 127.126.201.1:64030: connect: Connection refused (error 111) 20:42:08.271 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.270381 32759 heartbeater.cc:310] No valid master proxy. Connecting... 20:42:08.271 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.271157 32651 connection.cc:257] Transfer of RPC call RPC call kudu.master.MasterService.Ping -> {remote=127.126.201.1:64030, user_credentials={real_user=jenkins-slave, eff_user=}} aborted: Runtime error: RPC transfer destroyed before it finished sending 20:42:08.271 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) W0623 20:42:08.271256 32759 heartbeater.cc:402] Failed to heartbeat to 127.126.201.1:64030: Network error: Failed to ping master at 127.126.201.1:64030: Client connection negotiation failed: client connection to 127.126.201.1:64030: connect: Connection refused (error 111) 20:42:08.271 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.271327 32759 heartbeater.cc:310] No valid master proxy. Connecting... 20:42:08.272 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.272002 32651 connection.cc:257] Transfer of RPC call RPC call kudu.master.MasterService.Ping -> {remote=127.126.201.1:64030, user_credentials={real_user=jenkins-slave, eff_user=}} aborted: Runtime error: RPC transfer destroyed before it finished sending 20:42:08.272 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) W0623 20:42:08.272088 32759 heartbeater.cc:402] Failed to heartbeat to 127.126.201.1:64030: Network error: Failed to ping master at 127.126.201.1:64030: Client connection negotiation failed: client connection to 127.126.201.1:64030: connect: Connection refused (error 111) 20:42:08.272 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) W0623 20:42:08.272124 32759 heartbeater.cc:283] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts. 20:42:08.272 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.272169 32759 heartbeater.cc:310] No valid master proxy. Connecting... 20:42:08.272 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) I0623 20:42:08.272766 32651 connection.cc:257] Transfer of RPC call RPC call kudu.master.MasterService.Ping -> {remote=127.126.201.1:64030, user_credentials={real_user=jenkins-slave, eff_user=}} aborted: Runtime error: RPC transfer destroyed before it finished sending 20:42:08.273 [INFO - kudu-tserver:64034] (MiniKuduCluster.java:405) W0623 20:42:08.272850 32759 heartbeater.cc:402] Failed to heartbeat to 127.126.201.1:64030: Network error: Failed to ping master at 127.126.201.1:64030: Client connection negotiation failed: client connection to 127.126.201.1:64030: connect: Connection refused (error 111) 20:42:08.314 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.313987 32646 mem_tracker.cc:175] Creating tracker root 20:42:08.314 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.314301 32646 mem_tracker.cc:138] MemTracker: hard memory limit is 94.627029 GB 20:42:08.314 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.314347 32646 mem_tracker.cc:140] MemTracker: soft memory limit is 56.776218 GB 20:42:08.314 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.314586 32646 mem_tracker.cc:175] Creating tracker server->root 20:42:08.316 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.316205 32646 mem_tracker.cc:175] Creating tracker code_cache-sharded_lru_cache->root 20:42:08.320 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.320076 32646 tablet_server_main.cc:54] Initializing tablet server... 20:42:08.320 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.320261 32646 mem_tracker.cc:175] Creating tracker block_cache-sharded_lru_cache->root 20:42:08.321 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.321400 32646 hybrid_clock.cc:178] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 814229 20:42:08.321 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.321890 32646 fs_manager.cc:198] WAL root: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-2-1466714527141 20:42:08.322 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.321930 32646 fs_manager.cc:199] Metadata root: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-2-1466714527141 20:42:08.322 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.321962 32646 fs_manager.cc:200] Data roots: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-2-1466714527141 20:42:08.322 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.321981 32646 fs_manager.cc:201] All roots: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-2-1466714527141 20:42:08.322 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.322181 32646 mem_tracker.cc:175] Creating tracker log_block_manager->server->root 20:42:08.323 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.323889 32646 server_base.cc:164] Could not load existing FS layout: Not found: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-2-1466714527141/instance: No such file or directory (error 2) 20:42:08.324 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.323972 32646 server_base.cc:165] Creating new FS layout 20:42:08.327 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.327095 32646 fs_manager.cc:345] Generated new instance metadata in path /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-2-1466714527141/instance: 20:42:08.327 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) uuid: "04011d2931ac48ebb82c31f6686c1b26" 20:42:08.327 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) format_stamp: "Formatted at 2016-06-23 20:42:08 on jenkins-slave-c3oe" 20:42:08.328 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.328598 32646 pb_util.cc:258] Reading PB with version 2 starting at offset 16 20:42:08.328 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.328804 32646 pb_util.cc:258] Reading PB with version 2 starting at offset 555 20:42:08.330 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.330328 304 pb_util.cc:258] Reading PB with version 2 starting at offset 16 20:42:08.330 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.330513 304 pb_util.cc:258] Reading PB with version 2 starting at offset 559 20:42:08.331 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.331149 32646 fs_manager.cc:242] Opened local filesystem: /home/jenkins-slave/workspace/kudu-1/build/test-tmp/ts-2-1466714527141 20:42:08.331 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) uuid: "04011d2931ac48ebb82c31f6686c1b26" 20:42:08.331 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) format_stamp: "Formatted at 2016-06-23 20:42:08 on jenkins-slave-c3oe" 20:42:08.336 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.336235 32646 fs_manager.cc:401] Ignoring hidden file in tablet metadata dir: . 20:42:08.336 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.336285 32646 fs_manager.cc:401] Ignoring hidden file in tablet metadata dir: .. 20:42:08.336 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.336799 32646 tablet_server_main.cc:57] Starting tablet server... 20:42:08.365 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.365298 32646 rpc_server.cc:164] RPC server started. Bound to: 127.126.201.1:64036 20:42:08.365 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.365775 32646 webserver.cc:124] Starting webserver on 127.126.201.1:64037 20:42:08.365 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.365821 32646 webserver.cc:135] Document root disabled 20:42:08.366 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.366492 32646 webserver.cc:218] Webserver started. Bound to: http://127.126.201.1:64037/ 20:42:08.366 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.366863 414 heartbeater.cc:365] Heartbeat thread starting 20:42:08.367 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.366988 414 heartbeater.cc:310] No valid master proxy. Connecting... 20:42:08.367 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.367197 32646 tablet_server_main.cc:60] Tablet server successfully started. 20:42:08.414 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.414115 306 connection.cc:257] Transfer of RPC call RPC call kudu.master.MasterService.Ping -> {remote=127.126.201.1:64030, user_credentials={real_user=jenkins-slave, eff_user=}} aborted: Runtime error: RPC transfer destroyed before it finished sending 20:42:08.414 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) W0623 20:42:08.414325 414 heartbeater.cc:402] Failed to heartbeat to 127.126.201.1:64030: Network error: Failed to ping master at 127.126.201.1:64030: Client connection negotiation failed: client connection to 127.126.201.1:64030: connect: Connection refused (error 111) 20:42:08.414 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.414450 414 heartbeater.cc:310] No valid master proxy. Connecting... 20:42:08.415 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.415520 306 connection.cc:257] Transfer of RPC call RPC call kudu.master.MasterService.Ping -> {remote=127.126.201.1:64030, user_credentials={real_user=jenkins-slave, eff_user=}} aborted: Runtime error: RPC transfer destroyed before it finished sending 20:42:08.415 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) W0623 20:42:08.415598 414 heartbeater.cc:402] Failed to heartbeat to 127.126.201.1:64030: Network error: Failed to ping master at 127.126.201.1:64030: Client connection negotiation failed: client connection to 127.126.201.1:64030: connect: Connection refused (error 111) 20:42:08.415 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.415664 414 heartbeater.cc:310] No valid master proxy. Connecting... 20:42:08.416 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.416250 306 connection.cc:257] Transfer of RPC call RPC call kudu.master.MasterService.Ping -> {remote=127.126.201.1:64030, user_credentials={real_user=jenkins-slave, eff_user=}} aborted: Runtime error: RPC transfer destroyed before it finished sending 20:42:08.416 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) W0623 20:42:08.416314 414 heartbeater.cc:402] Failed to heartbeat to 127.126.201.1:64030: Network error: Failed to ping master at 127.126.201.1:64030: Client connection negotiation failed: client connection to 127.126.201.1:64030: connect: Connection refused (error 111) 20:42:08.416 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) W0623 20:42:08.416344 414 heartbeater.cc:283] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts. 20:42:08.416 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.416389 414 heartbeater.cc:310] No valid master proxy. Connecting... 20:42:08.417 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) I0623 20:42:08.417068 306 connection.cc:257] Transfer of RPC call RPC call kudu.master.MasterService.Ping -> {remote=127.126.201.1:64030, user_credentials={real_user=jenkins-slave, eff_user=}} aborted: Runtime error: RPC transfer destroyed before it finished sending 20:42:08.417 [INFO - kudu-tserver:64036] (MiniKuduCluster.java:405) W0623 20:42:08.417171 414 heartbeater.cc:402] Failed to heartbeat to 127.126.201.1:64030: Network error: Failed to ping master at 127.126.201.1:64030: Client connection negotiation failed: client connection to 127.126.201.1:64030: connect: Connection refused (error 111) 20:42:08.580 [INFO - main] (BaseKuduTest.java:87) Creating new Kudu client... 20:42:08.597 [INFO - main] (BaseKuduTest.java:92) Waiting for tablet servers... 20:42:08.846 [DEBUG - main] (TabletClient.java:633) [Peer Kudu Master - 127.126.201.1:64030] [id: 0x10e02648] OPEN 20:42:08.854 [DEBUG - New I/O boss #65] (AsyncKuduClient.java:1686) Removed from IP cache: {127.126.201.1:64030} -> {TabletClient@341074620(chan=null, uuid=Kudu Master - 127.126.201.1:64030, #pending_rpcs=1, #rpcs_inflight=0)} 20:42:08.856 [DEBUG - New I/O worker #1] (TabletClient.java:633) [Peer Kudu Master - 127.126.201.1:64030] [id: 0x10e02648] CLOSED 20:42:08.856 [DEBUG - New I/O boss #65] (TabletClient.java:633) [Peer Kudu Master - 127.126.201.1:64030] [id: 0x10e02648] EXCEPTION: java.net.ConnectException: Connection refused: /127.126.201.1:64030 20:42:08.856 [WARN - New I/O worker #1] (GetMasterRegistrationReceived.java:207) Error receiving a response from: 127.126.201.1:64030 org.kududb.client.ConnectionResetException: [Peer Kudu Master - 127.126.201.1:64030] Connection reset on [id: 0x10e02648] at org.kududb.client.TabletClient.cleanup(TabletClient.java:681) at org.kududb.client.TabletClient.channelClosed(TabletClient.java:654) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:88) at org.kududb.client.TabletClient.handleUpstream(TabletClient.java:635) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.timeout.ReadTimeoutHandler.channelClosed(ReadTimeoutHandler.java:176) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:88) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.kududb.client.AsyncKuduClient$TabletClientPipeline.sendUpstream(AsyncKuduClient.java:1763) at org.jboss.netty.channel.Channels.fireChannelClosed(Channels.java:468) at org.jboss.netty.channel.Channels$6.run(Channels.java:457) at org.jboss.netty.channel.socket.ChannelRunnableWrapper.run(ChannelRunnableWrapper.java:40) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 20:42:08.857 [ERROR - New I/O boss #65] (TabletClient.java:732) [Peer Kudu Master - 127.126.201.1:64030] Unexpected exception from downstream on [id: 0x10e02648] java.net.ConnectException: Connection refused: /127.126.201.1:64030 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152) at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 20:42:08.859 [WARN - New I/O worker #1] (GetMasterRegistrationReceived.java:121) Unable to find the leader master (127.126.201.1:64030), will retry 20:42:08.860 [DEBUG - New I/O worker #1] (AsyncKuduClient.java:947) Going to sleep for 1 at retry 1 20:42:08.862 [DEBUG - main] (AsyncKuduClient.java:1503) Releasing all remaining resources 20:42:08.867 [DEBUG - Hashed wheel timer #1] (TabletClient.java:633) [Peer Kudu Master - 127.126.201.1:64030] [id: 0x29a4c0a6] OPEN 20:42:08.868 [DEBUG - New I/O boss #65] (AsyncKuduClient.java:1686) Removed from IP cache: {127.126.201.1:64030} -> {TabletClient@1859177524(chan=null, uuid=Kudu Master - 127.126.201.1:64030, #pending_rpcs=1, #rpcs_inflight=0)} 20:42:08.869 [DEBUG - New I/O worker #2] (TabletClient.java:633) [Peer Kudu Master - 127.126.201.1:64030] [id: 0x29a4c0a6] CLOSED 20:42:08.869 [WARN - New I/O worker #2] (GetMasterRegistrationReceived.java:207) Error receiving a response from: 127.126.201.1:64030 org.kududb.client.ConnectionResetException: [Peer Kudu Master - 127.126.201.1:64030] Connection reset on [id: 0x29a4c0a6] at org.kududb.client.TabletClient.cleanup(TabletClient.java:681) at org.kududb.client.TabletClient.channelClosed(TabletClient.java:654) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:88) at org.kududb.client.TabletClient.handleUpstream(TabletClient.java:635) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.timeout.ReadTimeoutHandler.channelClosed(ReadTimeoutHandler.java:176) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:88) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.kududb.client.AsyncKuduClient$TabletClientPipeline.sendUpstream(AsyncKuduClient.java:1763) at org.jboss.netty.channel.Channels.fireChannelClosed(Channels.java:468) at org.jboss.netty.channel.Channels$6.run(Channels.java:457) at org.jboss.netty.channel.socket.ChannelRunnableWrapper.run(ChannelRunnableWrapper.java:40) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 20:42:08.870 [DEBUG - New I/O boss #65] (TabletClient.java:633) [Peer Kudu Master - 127.126.201.1:64030] [id: 0x29a4c0a6] EXCEPTION: java.net.ConnectException: Connection refused: /127.126.201.1:64030 20:42:08.870 [WARN - New I/O worker #2] (GetMasterRegistrationReceived.java:121) Unable to find the leader master (127.126.201.1:64030), will retry 20:42:08.870 [ERROR - New I/O boss #65] (TabletClient.java:732) [Peer Kudu Master - 127.126.201.1:64030] Unexpected exception from downstream on [id: 0x29a4c0a6] java.net.ConnectException: Connection refused: /127.126.201.1:64030 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152) at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 20:42:08.871 [DEBUG - New I/O worker #2] (AsyncKuduClient.java:947) Going to sleep for 3 at retry 2 20:42:08.885 [DEBUG - Hashed wheel timer #1] (TabletClient.java:633) [Peer Kudu Master - 127.126.201.1:64030] [id: 0x438ea5f0] OPEN 20:42:08.886 [DEBUG - New I/O boss #65] (AsyncKuduClient.java:1686) Removed from IP cache: {127.126.201.1:64030} -> {TabletClient@1708519117(chan=null, uuid=Kudu Master - 127.126.201.1:64030, #pending_rpcs=1, #rpcs_inflight=0)} 20:42:08.887 [DEBUG - New I/O boss #65] (TabletClient.java:633) [Peer Kudu Master - 127.126.201.1:64030] [id: 0x438ea5f0] EXCEPTION: java.net.ConnectException: Connection refused: /127.126.201.1:64030 20:42:08.887 [DEBUG - New I/O worker #3] (TabletClient.java:633) [Peer Kudu Master - 127.126.201.1:64030] [id: 0x438ea5f0] CLOSED 20:42:08.887 [ERROR - New I/O boss #65] (TabletClient.java:732) [Peer Kudu Master - 127.126.201.1:64030] Unexpected exception from downstream on [id: 0x438ea5f0] java.net.ConnectException: Connection refused: /127.126.201.1:64030 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152) at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 20:42:08.887 [WARN - New I/O worker #3] (GetMasterRegistrationReceived.java:207) Error receiving a response from: 127.126.201.1:64030 org.kududb.client.ConnectionResetException: [Peer Kudu Master - 127.126.201.1:64030] Connection reset on [id: 0x438ea5f0] at org.kududb.client.TabletClient.cleanup(TabletClient.java:681) at org.kududb.client.TabletClient.channelClosed(TabletClient.java:654) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:88) at org.kududb.client.TabletClient.handleUpstream(TabletClient.java:635) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.timeout.ReadTimeoutHandler.channelClosed(ReadTimeoutHandler.java:176) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:88) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.kududb.client.AsyncKuduClient$TabletClientPipeline.sendUpstream(AsyncKuduClient.java:1763) at org.jboss.netty.channel.Channels.fireChannelClosed(Channels.java:468) at org.jboss.netty.channel.Channels$6.run(Channels.java:457) at org.jboss.netty.channel.socket.ChannelRunnableWrapper.run(ChannelRunnableWrapper.java:40) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 20:42:08.889 [WARN - New I/O worker #3] (GetMasterRegistrationReceived.java:121) Unable to find the leader master (127.126.201.1:64030), will retry 20:42:08.889 [DEBUG - New I/O worker #3] (AsyncKuduClient.java:947) Going to sleep for 7 at retry 3