I0109 07:20:11.312049 1972 master_main.cc:60] Master server non-default flags: --fs_data_dirs=/var/lib/kudu/master --fs_wal_dir=/var/lib/kudu/master --heap_profile_path=/tmp/kudu-master.1972 --unlock_experimental_flags=true --unlock_unsafe_flags=true --flagfile=/etc/kudu/conf/master.gflagfile --log_dir=/var/log/kudu --logtostderr=true Master server version: kudu 1.3.0-SNAPSHOT revision 3a23b058af0f4de04f8827255de405fff7bf9ee1 build type RELEASE built by jenkins at 09 Jan 2017 06:07:18 PST on aserbin-kudu-jepsen-control-node-debian8.gce.cloudera.com build id 2017-01-09_06-07-06 I0109 07:20:11.358667 1972 mem_tracker.cc:140] MemTracker: hard memory limit is 0.464404 GB I0109 07:20:11.358731 1972 mem_tracker.cc:142] MemTracker: soft memory limit is 0.278643 GB I0109 07:20:11.423041 1972 master_main.cc:67] Initializing master server... I0109 07:20:11.423573 1972 hybrid_clock.cc:177] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 62691 I0109 07:20:11.425781 1972 env_posix.cc:1286] Not raising process file limit of 65536; it is already as high as it can go I0109 07:20:11.425812 1972 file_cache.cc:401] Constructed file cache lbm with capacity 32768 I0109 07:20:11.425981 1972 server_base.cc:179] Could not load existing FS layout: Not found: /var/lib/kudu/master/instance: No such file or directory (error 2) I0109 07:20:11.425997 1972 server_base.cc:180] Creating new FS layout I0109 07:20:11.433568 1972 fs_manager.cc:362] Generated new instance metadata in path /var/lib/kudu/master/instance: uuid: "161e818e715f4966ae9cb9a05ef6e3e1" format_stamp: "Formatted at 2017-01-09 15:20:11 on aserbin-kudu-jepsen-master" I0109 07:20:11.444437 1972 fs_manager.cc:251] Opened local filesystem: /var/lib/kudu/master uuid: "161e818e715f4966ae9cb9a05ef6e3e1" format_stamp: "Formatted at 2017-01-09 15:20:11 on aserbin-kudu-jepsen-master" I0109 07:20:11.448813 1972 master_main.cc:70] Starting Master server... I0109 07:20:11.459950 1972 rpc_server.cc:164] RPC server started. Bound to: 0.0.0.0:7051 I0109 07:20:11.460052 1972 webserver.cc:126] Starting webserver on 0.0.0.0:8051 I0109 07:20:11.460065 1972 webserver.cc:137] Document root disabled I0109 07:20:11.460266 1972 webserver.cc:221] Webserver started. Bound to: http://0.0.0.0:8051/ I0109 07:20:11.471791 2038 tablet_bootstrap.cc:381] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1: Bootstrap starting. I0109 07:20:11.474436 2038 tablet_bootstrap.cc:540] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I0109 07:20:11.474588 2038 tablet_bootstrap.cc:483] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1: No blocks or log segments found. Creating new log. I0109 07:20:11.474750 2038 log.cc:351] Log is configured to *not* fsync() on all Append() calls I0109 07:20:11.475311 2038 tablet_bootstrap.cc:381] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1: No bootstrap required, opened a new log I0109 07:20:11.475814 2038 raft_consensus.cc:287] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e818e715f4966ae9cb9a05ef6e3e1" member_type: VOTER } I0109 07:20:11.475859 2038 raft_consensus.cc:318] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0109 07:20:11.475873 2038 raft_consensus.cc:547] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 161e818e715f4966ae9cb9a05ef6e3e1, State: 1, Role: FOLLOWER I0109 07:20:11.475889 2038 consensus_queue.cc:175] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I0109 07:20:11.475908 2038 raft_consensus.cc:331] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0109 07:20:11.475919 2038 raft_consensus.cc:410] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0109 07:20:11.475934 2038 raft_consensus.cc:2158] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.933s I0109 07:20:11.475944 2038 raft_consensus.cc:2202] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [term 0 FOLLOWER]: Advancing to term 1 I0109 07:20:11.476227 2038 raft_consensus.cc:434] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e818e715f4966ae9cb9a05ef6e3e1" member_type: VOTER } I0109 07:20:11.476258 2038 leader_election.cc:242] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I0109 07:20:11.476474 2038 sys_catalog.cc:257] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletPeer. Latest consensus state: current_term: 1 config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e818e715f4966ae9cb9a05ef6e3e1" member_type: VOTER } } I0109 07:20:11.476490 2038 sys_catalog.cc:260] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [sys.catalog]: This master's current role is: FOLLOWER I0109 07:20:11.476541 2038 sys_catalog.cc:335] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [sys.catalog]: configured and running, proceeding with master startup. I0109 07:20:11.476625 2042 raft_consensus.cc:2158] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.157s I0109 07:20:11.476644 2042 raft_consensus.cc:1968] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [term 1 FOLLOWER]: Leader election won for term 1 I0109 07:20:11.476681 2042 raft_consensus.cc:517] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [term 1 LEADER]: Becoming Leader. State: Replica: 161e818e715f4966ae9cb9a05ef6e3e1, State: 1, Role: LEADER I0109 07:20:11.476740 2042 consensus_queue.cc:157] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 1, Majority size: 1, State: 1, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e818e715f4966ae9cb9a05ef6e3e1" member_type: VOTER } I0109 07:20:11.479064 1972 master_main.cc:73] Master server successfully started. I0109 07:20:11.479908 2042 sys_catalog.cc:257] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "161e818e715f4966ae9cb9a05ef6e3e1" config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e818e715f4966ae9cb9a05ef6e3e1" member_type: VOTER } } I0109 07:20:11.479961 2042 sys_catalog.cc:260] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [sys.catalog]: This master's current role is: LEADER I0109 07:20:11.480262 2042 sys_catalog.cc:257] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 161e818e715f4966ae9cb9a05ef6e3e1. Latest consensus state: current_term: 1 leader_uuid: "161e818e715f4966ae9cb9a05ef6e3e1" config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e818e715f4966ae9cb9a05ef6e3e1" member_type: VOTER } } I0109 07:20:11.480279 2042 sys_catalog.cc:260] T 00000000000000000000000000000000 P 161e818e715f4966ae9cb9a05ef6e3e1 [sys.catalog]: This master's current role is: LEADER I0109 07:20:11.480329 2045 catalog_manager.cc:708] Loading table and tablet metadata into memory... I0109 07:20:11.571566 2005 ts_manager.cc:76] Registered new tserver with Master: 1678276a36f849c28459de6b6788965a (aserbin-kudu-jepsen-ts-5.gce.cloudera.com:7050) I0109 07:20:11.679780 2005 ts_manager.cc:76] Registered new tserver with Master: c5a3e756029444b2bf66ee0044f5bc35 (aserbin-kudu-jepsen-ts-1.gce.cloudera.com:7050) I0109 07:20:12.067822 2005 ts_manager.cc:76] Registered new tserver with Master: 591b1a1d383d4b659d7549653f4f136e (aserbin-kudu-jepsen-ts-2.gce.cloudera.com:7050) I0109 07:20:12.352514 2005 ts_manager.cc:76] Registered new tserver with Master: 710c236995db46699eb9055c19ac506e (aserbin-kudu-jepsen-ts-4.gce.cloudera.com:7050) I0109 07:20:16.201686 2005 ts_manager.cc:76] Registered new tserver with Master: 7897578c54854ba0a5af32eb2aa04704 (aserbin-kudu-jepsen-ts-3.gce.cloudera.com:7050) I0109 07:20:16.671998 2005 catalog_manager.cc:917] CreateTable from {real_user=java_client} at 172.31.112.10:48491: name: "rw-register-1483975178870" schema { columns { name: "key" type: STRING is_key: true is_nullable: false cfile_block_size: 0 } columns { name: "value" type: INT32 is_key: false is_nullable: false cfile_block_size: 0 } } num_replicas: 5 partition_schema { range_schema { columns { name: "key" } } } I0109 07:20:17.658682 2004 catalog_manager.cc:2180] T 9e99b710634a44f4a948132a5a180600 reported consensus state change: term changed from 0 to 1, leader changed from to 710c236995db46699eb9055c19ac506e (aserbin-kudu-jepsen-ts-4.gce.cloudera.com). New consensus state: current_term: 1 leader_uuid: "710c236995db46699eb9055c19ac506e" config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710c236995db46699eb9055c19ac506e" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-4.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "1678276a36f849c28459de6b6788965a" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-5.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "591b1a1d383d4b659d7549653f4f136e" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-2.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "c5a3e756029444b2bf66ee0044f5bc35" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-1.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "7897578c54854ba0a5af32eb2aa04704" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-3.gce.cloudera.com" port: 7050 } } } I0109 07:20:30.361487 2004 ts_manager.cc:82] Re-registered known tserver with Master: 710c236995db46699eb9055c19ac506e (aserbin-kudu-jepsen-ts-4.gce.cloudera.com:7050) I0109 07:20:30.571378 2004 ts_manager.cc:82] Re-registered known tserver with Master: 7897578c54854ba0a5af32eb2aa04704 (aserbin-kudu-jepsen-ts-3.gce.cloudera.com:7050) I0109 07:20:30.821004 2004 ts_manager.cc:82] Re-registered known tserver with Master: 591b1a1d383d4b659d7549653f4f136e (aserbin-kudu-jepsen-ts-2.gce.cloudera.com:7050) I0109 07:20:31.122087 2004 ts_manager.cc:82] Re-registered known tserver with Master: c5a3e756029444b2bf66ee0044f5bc35 (aserbin-kudu-jepsen-ts-1.gce.cloudera.com:7050) I0109 07:20:31.316875 2004 ts_manager.cc:82] Re-registered known tserver with Master: 1678276a36f849c28459de6b6788965a (aserbin-kudu-jepsen-ts-5.gce.cloudera.com:7050) I0109 07:20:34.106477 2004 catalog_manager.cc:2180] T 9e99b710634a44f4a948132a5a180600 reported consensus state change: term changed from 1 to 4, leader changed from 710c236995db46699eb9055c19ac506e (aserbin-kudu-jepsen-ts-4.gce.cloudera.com) to 1678276a36f849c28459de6b6788965a (aserbin-kudu-jepsen-ts-5.gce.cloudera.com). New consensus state: current_term: 4 leader_uuid: "1678276a36f849c28459de6b6788965a" config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710c236995db46699eb9055c19ac506e" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-4.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "1678276a36f849c28459de6b6788965a" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-5.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "591b1a1d383d4b659d7549653f4f136e" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-2.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "c5a3e756029444b2bf66ee0044f5bc35" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-1.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "7897578c54854ba0a5af32eb2aa04704" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-3.gce.cloudera.com" port: 7050 } } } I0109 07:20:43.841171 2004 ts_manager.cc:82] Re-registered known tserver with Master: 591b1a1d383d4b659d7549653f4f136e (aserbin-kudu-jepsen-ts-2.gce.cloudera.com:7050) I0109 07:20:44.093516 2004 ts_manager.cc:82] Re-registered known tserver with Master: c5a3e756029444b2bf66ee0044f5bc35 (aserbin-kudu-jepsen-ts-1.gce.cloudera.com:7050) I0109 07:20:44.316231 2004 ts_manager.cc:82] Re-registered known tserver with Master: 710c236995db46699eb9055c19ac506e (aserbin-kudu-jepsen-ts-4.gce.cloudera.com:7050) I0109 07:20:44.547425 2004 ts_manager.cc:82] Re-registered known tserver with Master: 1678276a36f849c28459de6b6788965a (aserbin-kudu-jepsen-ts-5.gce.cloudera.com:7050) I0109 07:20:44.723951 2004 ts_manager.cc:82] Re-registered known tserver with Master: 7897578c54854ba0a5af32eb2aa04704 (aserbin-kudu-jepsen-ts-3.gce.cloudera.com:7050) I0109 07:20:45.103783 2004 catalog_manager.cc:2180] T 9e99b710634a44f4a948132a5a180600 reported consensus state change: term changed from 4 to 5, leader changed from 1678276a36f849c28459de6b6788965a (aserbin-kudu-jepsen-ts-5.gce.cloudera.com) to 710c236995db46699eb9055c19ac506e (aserbin-kudu-jepsen-ts-4.gce.cloudera.com). New consensus state: current_term: 5 leader_uuid: "710c236995db46699eb9055c19ac506e" config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710c236995db46699eb9055c19ac506e" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-4.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "1678276a36f849c28459de6b6788965a" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-5.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "591b1a1d383d4b659d7549653f4f136e" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-2.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "c5a3e756029444b2bf66ee0044f5bc35" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-1.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "7897578c54854ba0a5af32eb2aa04704" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-3.gce.cloudera.com" port: 7050 } } } I0109 07:21:06.633275 2004 ts_manager.cc:82] Re-registered known tserver with Master: 7897578c54854ba0a5af32eb2aa04704 (aserbin-kudu-jepsen-ts-3.gce.cloudera.com:7050) I0109 07:21:06.855053 2004 ts_manager.cc:82] Re-registered known tserver with Master: 710c236995db46699eb9055c19ac506e (aserbin-kudu-jepsen-ts-4.gce.cloudera.com:7050) I0109 07:21:07.076876 2004 ts_manager.cc:82] Re-registered known tserver with Master: c5a3e756029444b2bf66ee0044f5bc35 (aserbin-kudu-jepsen-ts-1.gce.cloudera.com:7050) I0109 07:21:07.324497 2004 ts_manager.cc:82] Re-registered known tserver with Master: 1678276a36f849c28459de6b6788965a (aserbin-kudu-jepsen-ts-5.gce.cloudera.com:7050) I0109 07:21:07.572976 2004 catalog_manager.cc:2180] T 9e99b710634a44f4a948132a5a180600 reported consensus state change: term changed from 5 to 6. New consensus state: current_term: 6 leader_uuid: "710c236995db46699eb9055c19ac506e" config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710c236995db46699eb9055c19ac506e" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-4.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "1678276a36f849c28459de6b6788965a" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-5.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "591b1a1d383d4b659d7549653f4f136e" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-2.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "c5a3e756029444b2bf66ee0044f5bc35" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-1.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "7897578c54854ba0a5af32eb2aa04704" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-3.gce.cloudera.com" port: 7050 } } } I0109 07:21:07.576483 2002 ts_manager.cc:82] Re-registered known tserver with Master: 591b1a1d383d4b659d7549653f4f136e (aserbin-kudu-jepsen-ts-2.gce.cloudera.com:7050) I0109 07:21:20.172092 2004 ts_manager.cc:82] Re-registered known tserver with Master: 7897578c54854ba0a5af32eb2aa04704 (aserbin-kudu-jepsen-ts-3.gce.cloudera.com:7050) I0109 07:21:20.411764 2004 ts_manager.cc:82] Re-registered known tserver with Master: 1678276a36f849c28459de6b6788965a (aserbin-kudu-jepsen-ts-5.gce.cloudera.com:7050) I0109 07:21:25.436923 2004 ts_manager.cc:82] Re-registered known tserver with Master: c5a3e756029444b2bf66ee0044f5bc35 (aserbin-kudu-jepsen-ts-1.gce.cloudera.com:7050) I0109 07:21:25.648892 2004 ts_manager.cc:82] Re-registered known tserver with Master: 591b1a1d383d4b659d7549653f4f136e (aserbin-kudu-jepsen-ts-2.gce.cloudera.com:7050) I0109 07:21:25.865841 2004 ts_manager.cc:82] Re-registered known tserver with Master: 710c236995db46699eb9055c19ac506e (aserbin-kudu-jepsen-ts-4.gce.cloudera.com:7050) I0109 07:21:26.210634 2004 catalog_manager.cc:2180] T 9e99b710634a44f4a948132a5a180600 reported consensus state change: term changed from 6 to 7, leader changed from 710c236995db46699eb9055c19ac506e (aserbin-kudu-jepsen-ts-4.gce.cloudera.com) to c5a3e756029444b2bf66ee0044f5bc35 (aserbin-kudu-jepsen-ts-1.gce.cloudera.com). New consensus state: current_term: 7 leader_uuid: "c5a3e756029444b2bf66ee0044f5bc35" config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710c236995db46699eb9055c19ac506e" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-4.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "1678276a36f849c28459de6b6788965a" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-5.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "591b1a1d383d4b659d7549653f4f136e" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-2.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "c5a3e756029444b2bf66ee0044f5bc35" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-1.gce.cloudera.com" port: 7050 } } peers { permanent_uuid: "7897578c54854ba0a5af32eb2aa04704" member_type: VOTER last_known_addr { host: "aserbin-kudu-jepsen-ts-3.gce.cloudera.com" port: 7050 } } }