[==========] Running 34 tests from 1 test case. [----------] Global test environment set-up. [----------] 34 tests from ClientTest [ RUN ] ClientTest.TestBadTable WARNING: Logging before InitGoogleLogging() is written to STDERR I1102 12:54:56.546373 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestBadTable.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "bf1144976a38486bbd43d12e7d6fd26c" format_stamp: "Formatted at 2014-11-02 20:54:56 on qiwi" I1102 12:54:56.546664 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestBadTable.1414961696528764-31819/minicluster-data/master-0-root uuid: "bf1144976a38486bbd43d12e7d6fd26c" format_stamp: "Formatted at 2014-11-02 20:54:56 on qiwi" I1102 12:54:56.549994 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 491278 I1102 12:54:56.555894 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:56.555912 31819 webserver.cc:117] Document root disabled I1102 12:54:56.556140 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:33158 I1102 12:54:56.557994 31849 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:54:56.560094 31849 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:54:56.560510 31849 log.cc:305] Log is configured to *not* fsync() on all Append() calls I1102 12:54:56.561486 31849 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:54:56.564224 31849 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:54:56.563864 31851 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: bf1144976a38486bbd43d12e7d6fd26c Changed from: seqno: 0 local: true peers { permanent_uuid: "bf1144976a38486bbd43d12e7d6fd26c" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "bf1144976a38486bbd43d12e7d6fd26c" role: LEADER } I1102 12:54:56.564451 31851 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "bf1144976a38486bbd43d12e7d6fd26c" role: LEADER } I1102 12:54:56.566337 31849 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:54:56.567389 31849 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:54:56.568503 31849 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:56.569231 31856 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: bf1144976a38486bbd43d12e7d6fd26c Changed from: seqno: 0 local: true peers { permanent_uuid: "bf1144976a38486bbd43d12e7d6fd26c" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "bf1144976a38486bbd43d12e7d6fd26c" role: LEADER } I1102 12:54:56.569308 31849 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:56.569308 31856 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "bf1144976a38486bbd43d12e7d6fd26c" role: LEADER } I1102 12:54:56.580549 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestBadTable.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "6f5fea57911d4b948191a043b6c53fac" format_stamp: "Formatted at 2014-11-02 20:54:56 on qiwi" I1102 12:54:56.580638 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestBadTable.1414961696528764-31819/minicluster-data/ts-0-root uuid: "6f5fea57911d4b948191a043b6c53fac" format_stamp: "Formatted at 2014-11-02 20:54:56 on qiwi" I1102 12:54:56.581886 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 491278 I1102 12:54:56.590224 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:56.590252 31819 webserver.cc:117] Document root disabled I1102 12:54:56.590490 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:57715 I1102 12:54:56.590863 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:54:56.594419 31899 heartbeater.cc:227] Registering TS with master... I1102 12:54:56.594506 31899 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:54:56.594960 31828 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "6f5fea57911d4b948191a043b6c53fac" instance_seqno: 1414961696589858 } with Master I1102 12:54:56.595752 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00437965s I1102 12:54:56.600450 31830 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:33062: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:54:56.608623 31877 tablet_service.cc:264] Processing CreateTablet for tablet 3ffe2b4288024717af6529bac174dd98 (table=client-testtb [id=e4fed1e2e56e4d8785dce9e3da0dfe29]), range=["\x00\x00\x00\t", ""] I1102 12:54:56.608649 31878 tablet_service.cc:264] Processing CreateTablet for tablet 12e2e2d79a914633bd898fb777afd1ac (table=client-testtb [id=e4fed1e2e56e4d8785dce9e3da0dfe29]), range=["", "\x00\x00\x00\t"] I1102 12:54:56.609838 31878 ts_tablet_manager.cc:460] Registered tablet 12e2e2d79a914633bd898fb777afd1ac I1102 12:54:56.609895 31877 ts_tablet_manager.cc:460] Registered tablet 3ffe2b4288024717af6529bac174dd98 I1102 12:54:56.610642 31909 ts_tablet_manager.cc:325] Bootstrapping tablet: 3ffe2b4288024717af6529bac174dd98 I1102 12:54:56.610648 31908 ts_tablet_manager.cc:325] Bootstrapping tablet: 12e2e2d79a914633bd898fb777afd1ac I1102 12:54:56.611029 31909 tablet_bootstrap.cc:234] Tablet 3ffe2b4288024717af6529bac174dd98: Bootstrap starting. I1102 12:54:56.611032 31908 tablet_bootstrap.cc:234] Tablet 12e2e2d79a914633bd898fb777afd1ac: Bootstrap starting. I1102 12:54:56.613286 31909 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 3ffe2b4288024717af6529bac174dd98 creating new one. I1102 12:54:56.613328 31908 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 12e2e2d79a914633bd898fb777afd1ac creating new one. I1102 12:54:56.615260 31908 tablet_bootstrap.cc:234] Tablet 12e2e2d79a914633bd898fb777afd1ac: No bootstrap required, opened a new log I1102 12:54:56.615298 31909 tablet_bootstrap.cc:234] Tablet 3ffe2b4288024717af6529bac174dd98: No bootstrap required, opened a new log I1102 12:54:56.615686 31908 ts_tablet_manager.cc:330] Time spent tablet 12e2e2d79a914633bd898fb777afd1ac bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:54:56.615684 31909 ts_tablet_manager.cc:330] Time spent tablet 3ffe2b4288024717af6529bac174dd98 bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:54:56.617547 31908 ts_tablet_manager.cc:350] Time spent starting tablet 12e2e2d79a914633bd898fb777afd1ac: real 0.002s user 0.001s sys 0.000s I1102 12:54:56.617648 31909 ts_tablet_manager.cc:350] Time spent starting tablet 3ffe2b4288024717af6529bac174dd98: real 0.002s user 0.001s sys 0.000s I1102 12:54:56.618803 31912 tablet_peer.cc:213] Configuration changed for tablet: 12e2e2d79a914633bd898fb777afd1ac in peer: 6f5fea57911d4b948191a043b6c53fac Changed from: seqno: -1 local: true peers { permanent_uuid: "6f5fea57911d4b948191a043b6c53fac" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 37172 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "6f5fea57911d4b948191a043b6c53fac" role: LEADER last_known_addr { host: "127.0.0.1" port: 37172 } } I1102 12:54:56.618814 31913 tablet_peer.cc:213] Configuration changed for tablet: 3ffe2b4288024717af6529bac174dd98 in peer: 6f5fea57911d4b948191a043b6c53fac Changed from: seqno: -1 local: true peers { permanent_uuid: "6f5fea57911d4b948191a043b6c53fac" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 37172 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "6f5fea57911d4b948191a043b6c53fac" role: LEADER last_known_addr { host: "127.0.0.1" port: 37172 } } I1102 12:54:56.627893 31830 catalog_manager.cc:1127] Tablet: 3ffe2b4288024717af6529bac174dd98 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "6f5fea57911d4b948191a043b6c53fac" role: LEADER last_known_addr { host: "127.0.0.1" port: 37172 } } I1102 12:54:56.630123 31830 catalog_manager.cc:1127] Tablet: 12e2e2d79a914633bd898fb777afd1ac reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "6f5fea57911d4b948191a043b6c53fac" role: LEADER last_known_addr { host: "127.0.0.1" port: 37172 } } I1102 12:54:56.639822 31835 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:33062: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:54:56.646512 31879 tablet_service.cc:264] Processing CreateTablet for tablet 723eb97fcc204d429170e97a7831d04c (table=client-testtb2 [id=75f37fe661de4750896dc2e7f72b4e57]), range=["", ""] I1102 12:54:56.647899 31879 ts_tablet_manager.cc:460] Registered tablet 723eb97fcc204d429170e97a7831d04c I1102 12:54:56.648012 31908 ts_tablet_manager.cc:325] Bootstrapping tablet: 723eb97fcc204d429170e97a7831d04c I1102 12:54:56.648586 31908 tablet_bootstrap.cc:234] Tablet 723eb97fcc204d429170e97a7831d04c: Bootstrap starting. I1102 12:54:56.654567 31908 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 723eb97fcc204d429170e97a7831d04c creating new one. I1102 12:54:56.658354 31908 tablet_bootstrap.cc:234] Tablet 723eb97fcc204d429170e97a7831d04c: No bootstrap required, opened a new log I1102 12:54:56.658967 31908 ts_tablet_manager.cc:330] Time spent tablet 723eb97fcc204d429170e97a7831d04c bootstrap: real 0.011s user 0.007s sys 0.002s I1102 12:54:56.662444 31908 ts_tablet_manager.cc:350] Time spent starting tablet 723eb97fcc204d429170e97a7831d04c: real 0.003s user 0.002s sys 0.001s I1102 12:54:56.663305 31919 tablet_peer.cc:213] Configuration changed for tablet: 723eb97fcc204d429170e97a7831d04c in peer: 6f5fea57911d4b948191a043b6c53fac Changed from: seqno: -1 local: true peers { permanent_uuid: "6f5fea57911d4b948191a043b6c53fac" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 37172 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "6f5fea57911d4b948191a043b6c53fac" role: LEADER last_known_addr { host: "127.0.0.1" port: 37172 } } I1102 12:54:56.666573 31834 catalog_manager.cc:1127] Tablet: 723eb97fcc204d429170e97a7831d04c reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "6f5fea57911d4b948191a043b6c53fac" role: LEADER last_known_addr { host: "127.0.0.1" port: 37172 } } I1102 12:54:56.677255 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:56.695859 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:54:56.696037 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:54:56.696441 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 3ffe2b4288024717af6529bac174dd98 I1102 12:54:56.696554 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(3ffe2b4288024717af6529bac174dd98) I1102 12:54:56.696630 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(3ffe2b4288024717af6529bac174dd98) I1102 12:54:56.696666 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(3ffe2b4288024717af6529bac174dd98) I1102 12:54:56.698456 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 12e2e2d79a914633bd898fb777afd1ac I1102 12:54:56.698528 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(12e2e2d79a914633bd898fb777afd1ac) I1102 12:54:56.698601 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(12e2e2d79a914633bd898fb777afd1ac) I1102 12:54:56.698639 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(12e2e2d79a914633bd898fb777afd1ac) I1102 12:54:56.700397 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 723eb97fcc204d429170e97a7831d04c I1102 12:54:56.700462 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(723eb97fcc204d429170e97a7831d04c) I1102 12:54:56.700516 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(723eb97fcc204d429170e97a7831d04c) I1102 12:54:56.700547 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(723eb97fcc204d429170e97a7831d04c) I1102 12:54:56.708953 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:56.708984 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:56.709041 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:56.710047 31819 master.cc:124] Master@127.0.0.1:40875 shutting down... I1102 12:54:56.721580 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:54:56.722064 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:54:56.722445 31819 master.cc:127] Master@127.0.0.1:40875 shutdown complete. [ OK ] ClientTest.TestBadTable (193 ms) [ RUN ] ClientTest.TestMasterDown I1102 12:54:56.867841 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestMasterDown.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "f3886384107f471c9587b2f9840e96e1" format_stamp: "Formatted at 2014-11-02 20:54:56 on qiwi" I1102 12:54:56.867938 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestMasterDown.1414961696528764-31819/minicluster-data/master-0-root uuid: "f3886384107f471c9587b2f9840e96e1" format_stamp: "Formatted at 2014-11-02 20:54:56 on qiwi" I1102 12:54:56.868768 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 491278 I1102 12:54:56.871661 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:56.871683 31819 webserver.cc:117] Document root disabled I1102 12:54:56.871906 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:60428 I1102 12:54:56.873236 31947 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:54:56.875087 31947 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:54:56.876009 31947 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:54:56.877465 31949 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: f3886384107f471c9587b2f9840e96e1 Changed from: seqno: 0 local: true peers { permanent_uuid: "f3886384107f471c9587b2f9840e96e1" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "f3886384107f471c9587b2f9840e96e1" role: LEADER } I1102 12:54:56.877523 31949 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "f3886384107f471c9587b2f9840e96e1" role: LEADER } I1102 12:54:56.877955 31947 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:54:56.880023 31947 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:54:56.880925 31947 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:54:56.882004 31947 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:56.882381 31953 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: f3886384107f471c9587b2f9840e96e1 Changed from: seqno: 0 local: true peers { permanent_uuid: "f3886384107f471c9587b2f9840e96e1" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "f3886384107f471c9587b2f9840e96e1" role: LEADER } I1102 12:54:56.882443 31947 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:56.882444 31953 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "f3886384107f471c9587b2f9840e96e1" role: LEADER } I1102 12:54:56.896625 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestMasterDown.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "3e1256dbfd2f4d55abf0b1b6004f40e3" format_stamp: "Formatted at 2014-11-02 20:54:56 on qiwi" I1102 12:54:56.896723 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestMasterDown.1414961696528764-31819/minicluster-data/ts-0-root uuid: "3e1256dbfd2f4d55abf0b1b6004f40e3" format_stamp: "Formatted at 2014-11-02 20:54:56 on qiwi" I1102 12:54:56.897570 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 491278 I1102 12:54:56.902339 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:56.902359 31819 webserver.cc:117] Document root disabled I1102 12:54:56.902592 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:53267 I1102 12:54:56.902720 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:54:56.904084 31996 heartbeater.cc:227] Registering TS with master... I1102 12:54:56.904134 31996 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:54:56.904455 31926 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "3e1256dbfd2f4d55abf0b1b6004f40e3" instance_seqno: 1414961696902062 } with Master I1102 12:54:56.905056 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00216373s I1102 12:54:56.911201 31928 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:52816: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:54:56.921746 31974 tablet_service.cc:264] Processing CreateTablet for tablet d8eb477c8e0846e19a5ba3f8e0ac6fd6 (table=client-testtb [id=62c3beaff7554aa79a8dc983a89b4b12]), range=["", "\x00\x00\x00\t"] I1102 12:54:56.921892 31975 tablet_service.cc:264] Processing CreateTablet for tablet babc6cff98ff4c72ac01a0af3fba87d0 (table=client-testtb [id=62c3beaff7554aa79a8dc983a89b4b12]), range=["\x00\x00\x00\t", ""] I1102 12:54:56.923378 31974 ts_tablet_manager.cc:460] Registered tablet d8eb477c8e0846e19a5ba3f8e0ac6fd6 I1102 12:54:56.923554 31975 ts_tablet_manager.cc:460] Registered tablet babc6cff98ff4c72ac01a0af3fba87d0 I1102 12:54:56.924135 32005 ts_tablet_manager.cc:325] Bootstrapping tablet: d8eb477c8e0846e19a5ba3f8e0ac6fd6 I1102 12:54:56.924156 32006 ts_tablet_manager.cc:325] Bootstrapping tablet: babc6cff98ff4c72ac01a0af3fba87d0 I1102 12:54:56.924834 32006 tablet_bootstrap.cc:234] Tablet babc6cff98ff4c72ac01a0af3fba87d0: Bootstrap starting. I1102 12:54:56.924865 32005 tablet_bootstrap.cc:234] Tablet d8eb477c8e0846e19a5ba3f8e0ac6fd6: Bootstrap starting. I1102 12:54:56.927839 32006 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: babc6cff98ff4c72ac01a0af3fba87d0 creating new one. I1102 12:54:56.927882 32005 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: d8eb477c8e0846e19a5ba3f8e0ac6fd6 creating new one. I1102 12:54:56.929313 32006 tablet_bootstrap.cc:234] Tablet babc6cff98ff4c72ac01a0af3fba87d0: No bootstrap required, opened a new log I1102 12:54:56.929406 32005 tablet_bootstrap.cc:234] Tablet d8eb477c8e0846e19a5ba3f8e0ac6fd6: No bootstrap required, opened a new log I1102 12:54:56.929553 32006 ts_tablet_manager.cc:330] Time spent tablet babc6cff98ff4c72ac01a0af3fba87d0 bootstrap: real 0.005s user 0.004s sys 0.001s I1102 12:54:56.929648 32005 ts_tablet_manager.cc:330] Time spent tablet d8eb477c8e0846e19a5ba3f8e0ac6fd6 bootstrap: real 0.005s user 0.004s sys 0.001s I1102 12:54:56.930850 32006 ts_tablet_manager.cc:350] Time spent starting tablet babc6cff98ff4c72ac01a0af3fba87d0: real 0.001s user 0.001s sys 0.000s I1102 12:54:56.930865 32005 ts_tablet_manager.cc:350] Time spent starting tablet d8eb477c8e0846e19a5ba3f8e0ac6fd6: real 0.001s user 0.001s sys 0.000s I1102 12:54:56.931514 32009 tablet_peer.cc:213] Configuration changed for tablet: babc6cff98ff4c72ac01a0af3fba87d0 in peer: 3e1256dbfd2f4d55abf0b1b6004f40e3 Changed from: seqno: -1 local: true peers { permanent_uuid: "3e1256dbfd2f4d55abf0b1b6004f40e3" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 44910 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "3e1256dbfd2f4d55abf0b1b6004f40e3" role: LEADER last_known_addr { host: "127.0.0.1" port: 44910 } } I1102 12:54:56.931526 32010 tablet_peer.cc:213] Configuration changed for tablet: d8eb477c8e0846e19a5ba3f8e0ac6fd6 in peer: 3e1256dbfd2f4d55abf0b1b6004f40e3 Changed from: seqno: -1 local: true peers { permanent_uuid: "3e1256dbfd2f4d55abf0b1b6004f40e3" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 44910 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "3e1256dbfd2f4d55abf0b1b6004f40e3" role: LEADER last_known_addr { host: "127.0.0.1" port: 44910 } } I1102 12:54:56.937237 31929 catalog_manager.cc:1127] Tablet: d8eb477c8e0846e19a5ba3f8e0ac6fd6 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "3e1256dbfd2f4d55abf0b1b6004f40e3" role: LEADER last_known_addr { host: "127.0.0.1" port: 44910 } } I1102 12:54:56.938822 31929 catalog_manager.cc:1127] Tablet: babc6cff98ff4c72ac01a0af3fba87d0 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "3e1256dbfd2f4d55abf0b1b6004f40e3" role: LEADER last_known_addr { host: "127.0.0.1" port: 44910 } } I1102 12:54:56.943914 31932 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:52816: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:54:56.948668 31976 tablet_service.cc:264] Processing CreateTablet for tablet 4181ef9877f24f96815ec46962721baa (table=client-testtb2 [id=86fefaaba1374b06809139aeb2f63d07]), range=["", ""] I1102 12:54:56.950309 31976 ts_tablet_manager.cc:460] Registered tablet 4181ef9877f24f96815ec46962721baa I1102 12:54:56.950502 32006 ts_tablet_manager.cc:325] Bootstrapping tablet: 4181ef9877f24f96815ec46962721baa I1102 12:54:56.951447 32006 tablet_bootstrap.cc:234] Tablet 4181ef9877f24f96815ec46962721baa: Bootstrap starting. I1102 12:54:56.956308 32006 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 4181ef9877f24f96815ec46962721baa creating new one. I1102 12:54:56.958416 32006 tablet_bootstrap.cc:234] Tablet 4181ef9877f24f96815ec46962721baa: No bootstrap required, opened a new log I1102 12:54:56.958868 32006 ts_tablet_manager.cc:330] Time spent tablet 4181ef9877f24f96815ec46962721baa bootstrap: real 0.008s user 0.006s sys 0.001s I1102 12:54:56.961235 32006 ts_tablet_manager.cc:350] Time spent starting tablet 4181ef9877f24f96815ec46962721baa: real 0.002s user 0.002s sys 0.001s I1102 12:54:56.961520 32016 tablet_peer.cc:213] Configuration changed for tablet: 4181ef9877f24f96815ec46962721baa in peer: 3e1256dbfd2f4d55abf0b1b6004f40e3 Changed from: seqno: -1 local: true peers { permanent_uuid: "3e1256dbfd2f4d55abf0b1b6004f40e3" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 44910 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "3e1256dbfd2f4d55abf0b1b6004f40e3" role: LEADER last_known_addr { host: "127.0.0.1" port: 44910 } } I1102 12:54:56.962338 31928 catalog_manager.cc:1127] Tablet: 4181ef9877f24f96815ec46962721baa reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "3e1256dbfd2f4d55abf0b1b6004f40e3" role: LEADER last_known_addr { host: "127.0.0.1" port: 44910 } } I1102 12:54:56.967968 31819 master.cc:124] Master@127.0.0.1:46633 shutting down... I1102 12:54:56.979756 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:54:56.980240 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:54:56.980634 31819 master.cc:127] Master@127.0.0.1:46633 shutdown complete. I1102 12:54:56.985267 31922 messenger.cc:190] Service unavailable: RPC Service is not registered: Call TSHeartbeat from 127.0.0.1:52815 (request call id 9) W1102 12:54:56.985522 31996 heartbeater.cc:277] Failed to heartbeat: Remote error: Failed to send heartbeat: Service unavailable: RPC Service is not registered: Call TSHeartbeat from 127.0.0.1:52815 (request call id 9) W1102 12:54:56.986449 32004 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:46633: connect: Connection refused (error 111) I1102 12:54:56.986593 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:57.003021 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:54:57.003192 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:54:57.003494 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: d8eb477c8e0846e19a5ba3f8e0ac6fd6 I1102 12:54:57.003571 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(d8eb477c8e0846e19a5ba3f8e0ac6fd6) I1102 12:54:57.003631 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(d8eb477c8e0846e19a5ba3f8e0ac6fd6) I1102 12:54:57.003679 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(d8eb477c8e0846e19a5ba3f8e0ac6fd6) I1102 12:54:57.005053 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 4181ef9877f24f96815ec46962721baa I1102 12:54:57.005162 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(4181ef9877f24f96815ec46962721baa) I1102 12:54:57.005234 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(4181ef9877f24f96815ec46962721baa) I1102 12:54:57.005266 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(4181ef9877f24f96815ec46962721baa) I1102 12:54:57.006459 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: babc6cff98ff4c72ac01a0af3fba87d0 I1102 12:54:57.006538 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(babc6cff98ff4c72ac01a0af3fba87d0) I1102 12:54:57.006598 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(babc6cff98ff4c72ac01a0af3fba87d0) I1102 12:54:57.006630 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(babc6cff98ff4c72ac01a0af3fba87d0) I1102 12:54:57.018637 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:57.018685 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:57.018774 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! [ OK ] ClientTest.TestMasterDown (296 ms) [ RUN ] ClientTest.TestScan I1102 12:54:57.174342 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScan.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "46f139ca20e44cd8940860edca57cc39" format_stamp: "Formatted at 2014-11-02 20:54:57 on qiwi" I1102 12:54:57.174747 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScan.1414961696528764-31819/minicluster-data/master-0-root uuid: "46f139ca20e44cd8940860edca57cc39" format_stamp: "Formatted at 2014-11-02 20:54:57 on qiwi" I1102 12:54:57.177319 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 491778 I1102 12:54:57.186082 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:57.186106 31819 webserver.cc:117] Document root disabled I1102 12:54:57.186419 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:43508 I1102 12:54:57.188148 32044 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:54:57.190788 32044 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:54:57.191797 32044 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:54:57.193487 32046 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 46f139ca20e44cd8940860edca57cc39 Changed from: seqno: 0 local: true peers { permanent_uuid: "46f139ca20e44cd8940860edca57cc39" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "46f139ca20e44cd8940860edca57cc39" role: LEADER } I1102 12:54:57.193552 32046 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "46f139ca20e44cd8940860edca57cc39" role: LEADER } I1102 12:54:57.194035 32044 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:54:57.196372 32044 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:54:57.197350 32044 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:54:57.198511 32044 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:57.198967 32050 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 46f139ca20e44cd8940860edca57cc39 Changed from: seqno: 0 local: true peers { permanent_uuid: "46f139ca20e44cd8940860edca57cc39" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "46f139ca20e44cd8940860edca57cc39" role: LEADER } I1102 12:54:57.199048 32044 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:57.199048 32050 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "46f139ca20e44cd8940860edca57cc39" role: LEADER } I1102 12:54:57.212267 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScan.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "abde96c5b6a7421a82891e50fc25863f" format_stamp: "Formatted at 2014-11-02 20:54:57 on qiwi" I1102 12:54:57.212391 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScan.1414961696528764-31819/minicluster-data/ts-0-root uuid: "abde96c5b6a7421a82891e50fc25863f" format_stamp: "Formatted at 2014-11-02 20:54:57 on qiwi" I1102 12:54:57.213459 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 491778 I1102 12:54:57.222501 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:57.222535 31819 webserver.cc:117] Document root disabled I1102 12:54:57.223021 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:34403 I1102 12:54:57.223300 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:54:57.226100 32093 heartbeater.cc:227] Registering TS with master... I1102 12:54:57.226182 32093 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:54:57.226706 32023 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "abde96c5b6a7421a82891e50fc25863f" instance_seqno: 1414961697221945 } with Master I1102 12:54:57.226990 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00331291s I1102 12:54:57.231497 32025 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:60892: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:54:57.238149 32071 tablet_service.cc:264] Processing CreateTablet for tablet 1f4a83136a6a4f7e94736fef28ca4ff6 (table=client-testtb [id=8e588b06afa24bfd98df2e9abd120380]), range=["\x00\x00\x00\t", ""] I1102 12:54:57.238181 32072 tablet_service.cc:264] Processing CreateTablet for tablet 1e20a91e1ec34c3da222639fc58c3b47 (table=client-testtb [id=8e588b06afa24bfd98df2e9abd120380]), range=["", "\x00\x00\x00\t"] I1102 12:54:57.239044 32072 ts_tablet_manager.cc:460] Registered tablet 1e20a91e1ec34c3da222639fc58c3b47 I1102 12:54:57.239081 32071 ts_tablet_manager.cc:460] Registered tablet 1f4a83136a6a4f7e94736fef28ca4ff6 I1102 12:54:57.239547 32105 ts_tablet_manager.cc:325] Bootstrapping tablet: 1f4a83136a6a4f7e94736fef28ca4ff6 I1102 12:54:57.239558 32104 ts_tablet_manager.cc:325] Bootstrapping tablet: 1e20a91e1ec34c3da222639fc58c3b47 I1102 12:54:57.239905 32105 tablet_bootstrap.cc:234] Tablet 1f4a83136a6a4f7e94736fef28ca4ff6: Bootstrap starting. I1102 12:54:57.239948 32104 tablet_bootstrap.cc:234] Tablet 1e20a91e1ec34c3da222639fc58c3b47: Bootstrap starting. I1102 12:54:57.242046 32105 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 1f4a83136a6a4f7e94736fef28ca4ff6 creating new one. I1102 12:54:57.242118 32104 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 1e20a91e1ec34c3da222639fc58c3b47 creating new one. I1102 12:54:57.243137 32104 tablet_bootstrap.cc:234] Tablet 1e20a91e1ec34c3da222639fc58c3b47: No bootstrap required, opened a new log I1102 12:54:57.243149 32105 tablet_bootstrap.cc:234] Tablet 1f4a83136a6a4f7e94736fef28ca4ff6: No bootstrap required, opened a new log I1102 12:54:57.243335 32105 ts_tablet_manager.cc:330] Time spent tablet 1f4a83136a6a4f7e94736fef28ca4ff6 bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:54:57.243340 32104 ts_tablet_manager.cc:330] Time spent tablet 1e20a91e1ec34c3da222639fc58c3b47 bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:54:57.244340 32104 ts_tablet_manager.cc:350] Time spent starting tablet 1e20a91e1ec34c3da222639fc58c3b47: real 0.001s user 0.001s sys 0.000s I1102 12:54:57.244417 32105 ts_tablet_manager.cc:350] Time spent starting tablet 1f4a83136a6a4f7e94736fef28ca4ff6: real 0.001s user 0.000s sys 0.000s I1102 12:54:57.244818 32109 tablet_peer.cc:213] Configuration changed for tablet: 1f4a83136a6a4f7e94736fef28ca4ff6 in peer: abde96c5b6a7421a82891e50fc25863f Changed from: seqno: -1 local: true peers { permanent_uuid: "abde96c5b6a7421a82891e50fc25863f" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 43538 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "abde96c5b6a7421a82891e50fc25863f" role: LEADER last_known_addr { host: "127.0.0.1" port: 43538 } } I1102 12:54:57.244830 32108 tablet_peer.cc:213] Configuration changed for tablet: 1e20a91e1ec34c3da222639fc58c3b47 in peer: abde96c5b6a7421a82891e50fc25863f Changed from: seqno: -1 local: true peers { permanent_uuid: "abde96c5b6a7421a82891e50fc25863f" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 43538 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "abde96c5b6a7421a82891e50fc25863f" role: LEADER last_known_addr { host: "127.0.0.1" port: 43538 } } I1102 12:54:57.248318 32022 catalog_manager.cc:1127] Tablet: 1f4a83136a6a4f7e94736fef28ca4ff6 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "abde96c5b6a7421a82891e50fc25863f" role: LEADER last_known_addr { host: "127.0.0.1" port: 43538 } } I1102 12:54:57.249886 32022 catalog_manager.cc:1127] Tablet: 1e20a91e1ec34c3da222639fc58c3b47 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "abde96c5b6a7421a82891e50fc25863f" role: LEADER last_known_addr { host: "127.0.0.1" port: 43538 } } I1102 12:54:57.255364 32027 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:60892: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:54:57.259775 32073 tablet_service.cc:264] Processing CreateTablet for tablet 44e7a16e9e45486aa4a80cee0223bad2 (table=client-testtb2 [id=7e171a7697344f8c9e86d1ec5ac85289]), range=["", ""] I1102 12:54:57.260669 32073 ts_tablet_manager.cc:460] Registered tablet 44e7a16e9e45486aa4a80cee0223bad2 I1102 12:54:57.260749 32104 ts_tablet_manager.cc:325] Bootstrapping tablet: 44e7a16e9e45486aa4a80cee0223bad2 I1102 12:54:57.261134 32104 tablet_bootstrap.cc:234] Tablet 44e7a16e9e45486aa4a80cee0223bad2: Bootstrap starting. I1102 12:54:57.264601 32104 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 44e7a16e9e45486aa4a80cee0223bad2 creating new one. I1102 12:54:57.266002 32104 tablet_bootstrap.cc:234] Tablet 44e7a16e9e45486aa4a80cee0223bad2: No bootstrap required, opened a new log I1102 12:54:57.266414 32104 ts_tablet_manager.cc:330] Time spent tablet 44e7a16e9e45486aa4a80cee0223bad2 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:54:57.267936 32104 ts_tablet_manager.cc:350] Time spent starting tablet 44e7a16e9e45486aa4a80cee0223bad2: real 0.001s user 0.001s sys 0.000s I1102 12:54:57.268163 32119 tablet_peer.cc:213] Configuration changed for tablet: 44e7a16e9e45486aa4a80cee0223bad2 in peer: abde96c5b6a7421a82891e50fc25863f Changed from: seqno: -1 local: true peers { permanent_uuid: "abde96c5b6a7421a82891e50fc25863f" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 43538 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "abde96c5b6a7421a82891e50fc25863f" role: LEADER last_known_addr { host: "127.0.0.1" port: 43538 } } I1102 12:54:57.272405 32025 catalog_manager.cc:1127] Tablet: 44e7a16e9e45486aa4a80cee0223bad2 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "abde96c5b6a7421a82891e50fc25863f" role: LEADER last_known_addr { host: "127.0.0.1" port: 43538 } } I1102 12:54:57.440198 31819 client-test.cc:224] Time spent Scanning with no predicates: real 0.019s user 0.007s sys 0.002s I1102 12:54:57.458053 31819 client-test.cc:255] Time spent Scanning with string predicate: real 0.018s user 0.002s sys 0.000s I1102 12:54:57.462185 31819 client-test.cc:282] Time spent Scanning with key predicate: real 0.004s user 0.001s sys 0.000s I1102 12:54:57.592526 31819 client-test.cc:282] Time spent Scanning with key predicate: real 0.007s user 0.001s sys 0.000s I1102 12:54:57.651243 31819 client-test.cc:282] Time spent Scanning with key predicate: real 0.004s user 0.001s sys 0.000s I1102 12:54:57.716843 31819 client-test.cc:282] Time spent Scanning with key predicate: real 0.004s user 0.001s sys 0.000s I1102 12:54:57.722460 31819 client-test.cc:282] Time spent Scanning with key predicate: real 0.003s user 0.001s sys 0.000s I1102 12:54:57.722534 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:57.741718 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:54:57.741832 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:54:57.742039 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 44e7a16e9e45486aa4a80cee0223bad2 I1102 12:54:57.742108 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(44e7a16e9e45486aa4a80cee0223bad2) I1102 12:54:57.742152 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(44e7a16e9e45486aa4a80cee0223bad2) I1102 12:54:57.742177 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(44e7a16e9e45486aa4a80cee0223bad2) I1102 12:54:57.743211 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 1f4a83136a6a4f7e94736fef28ca4ff6 I1102 12:54:57.743259 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(1f4a83136a6a4f7e94736fef28ca4ff6) I1102 12:54:57.743299 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(1f4a83136a6a4f7e94736fef28ca4ff6) I1102 12:54:57.743324 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(1f4a83136a6a4f7e94736fef28ca4ff6) I1102 12:54:57.744011 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 1e20a91e1ec34c3da222639fc58c3b47 I1102 12:54:57.744056 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(1e20a91e1ec34c3da222639fc58c3b47) I1102 12:54:57.744093 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(1e20a91e1ec34c3da222639fc58c3b47) I1102 12:54:57.744122 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(1e20a91e1ec34c3da222639fc58c3b47) I1102 12:54:57.752421 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:57.752461 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:57.752531 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:57.753578 31819 master.cc:124] Master@127.0.0.1:46577 shutting down... I1102 12:54:57.765017 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:54:57.765864 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:54:57.766851 31819 master.cc:127] Master@127.0.0.1:46577 shutdown complete. [ OK ] ClientTest.TestScan (759 ms) [ RUN ] ClientTest.TestScanAtSnapshot I1102 12:54:57.923948 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanAtSnapshot.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "5c581d96ac044c878ac24629238233e6" format_stamp: "Formatted at 2014-11-02 20:54:57 on qiwi" I1102 12:54:57.924140 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanAtSnapshot.1414961696528764-31819/minicluster-data/master-0-root uuid: "5c581d96ac044c878ac24629238233e6" format_stamp: "Formatted at 2014-11-02 20:54:57 on qiwi" I1102 12:54:57.925323 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 491778 I1102 12:54:57.930834 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:57.930869 31819 webserver.cc:117] Document root disabled I1102 12:54:57.931231 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:51361 I1102 12:54:57.933375 32152 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:54:57.935464 32152 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:54:57.936388 32152 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:54:57.937907 32154 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 5c581d96ac044c878ac24629238233e6 Changed from: seqno: 0 local: true peers { permanent_uuid: "5c581d96ac044c878ac24629238233e6" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "5c581d96ac044c878ac24629238233e6" role: LEADER } I1102 12:54:57.937968 32154 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "5c581d96ac044c878ac24629238233e6" role: LEADER } I1102 12:54:57.938316 32152 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:54:57.940377 32152 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:54:57.941273 32152 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:54:57.942308 32152 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:57.942716 32158 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 5c581d96ac044c878ac24629238233e6 Changed from: seqno: 0 local: true peers { permanent_uuid: "5c581d96ac044c878ac24629238233e6" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "5c581d96ac044c878ac24629238233e6" role: LEADER } I1102 12:54:57.942767 32152 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:57.942780 32158 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "5c581d96ac044c878ac24629238233e6" role: LEADER } I1102 12:54:57.954496 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanAtSnapshot.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "10c286ffa08f437abafebad3c325bf4f" format_stamp: "Formatted at 2014-11-02 20:54:57 on qiwi" I1102 12:54:57.954599 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanAtSnapshot.1414961696528764-31819/minicluster-data/ts-0-root uuid: "10c286ffa08f437abafebad3c325bf4f" format_stamp: "Formatted at 2014-11-02 20:54:57 on qiwi" I1102 12:54:57.955474 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 491778 I1102 12:54:57.959959 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:57.959976 31819 webserver.cc:117] Document root disabled I1102 12:54:57.960213 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:32836 I1102 12:54:57.960350 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:54:57.961891 32201 heartbeater.cc:227] Registering TS with master... I1102 12:54:57.961946 32201 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:54:57.962323 32131 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "10c286ffa08f437abafebad3c325bf4f" instance_seqno: 1414961697959671 } with Master I1102 12:54:57.962637 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00216757s I1102 12:54:57.965711 32133 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:44480: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:54:57.971194 32179 tablet_service.cc:264] Processing CreateTablet for tablet b4528c665a3149a9ac63fe038c159a71 (table=client-testtb [id=148c3035cafe4284990b9d4ad21888c5]), range=["", "\x00\x00\x00\t"] I1102 12:54:57.971225 32180 tablet_service.cc:264] Processing CreateTablet for tablet 47353cc3163440698639e9c90f0176b1 (table=client-testtb [id=148c3035cafe4284990b9d4ad21888c5]), range=["\x00\x00\x00\t", ""] I1102 12:54:57.972121 32180 ts_tablet_manager.cc:460] Registered tablet 47353cc3163440698639e9c90f0176b1 I1102 12:54:57.972157 32179 ts_tablet_manager.cc:460] Registered tablet b4528c665a3149a9ac63fe038c159a71 I1102 12:54:57.972549 32210 ts_tablet_manager.cc:325] Bootstrapping tablet: 47353cc3163440698639e9c90f0176b1 I1102 12:54:57.972569 32211 ts_tablet_manager.cc:325] Bootstrapping tablet: b4528c665a3149a9ac63fe038c159a71 I1102 12:54:57.972964 32210 tablet_bootstrap.cc:234] Tablet 47353cc3163440698639e9c90f0176b1: Bootstrap starting. I1102 12:54:57.973098 32211 tablet_bootstrap.cc:234] Tablet b4528c665a3149a9ac63fe038c159a71: Bootstrap starting. I1102 12:54:57.975823 32210 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 47353cc3163440698639e9c90f0176b1 creating new one. I1102 12:54:57.976028 32211 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: b4528c665a3149a9ac63fe038c159a71 creating new one. I1102 12:54:57.977149 32210 tablet_bootstrap.cc:234] Tablet 47353cc3163440698639e9c90f0176b1: No bootstrap required, opened a new log I1102 12:54:57.977159 32211 tablet_bootstrap.cc:234] Tablet b4528c665a3149a9ac63fe038c159a71: No bootstrap required, opened a new log I1102 12:54:57.977411 32211 ts_tablet_manager.cc:330] Time spent tablet b4528c665a3149a9ac63fe038c159a71 bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:54:57.977413 32210 ts_tablet_manager.cc:330] Time spent tablet 47353cc3163440698639e9c90f0176b1 bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:54:57.979120 32211 ts_tablet_manager.cc:350] Time spent starting tablet b4528c665a3149a9ac63fe038c159a71: real 0.002s user 0.001s sys 0.000s I1102 12:54:57.979228 32210 ts_tablet_manager.cc:350] Time spent starting tablet 47353cc3163440698639e9c90f0176b1: real 0.002s user 0.000s sys 0.000s I1102 12:54:57.979796 32215 tablet_peer.cc:213] Configuration changed for tablet: b4528c665a3149a9ac63fe038c159a71 in peer: 10c286ffa08f437abafebad3c325bf4f Changed from: seqno: -1 local: true peers { permanent_uuid: "10c286ffa08f437abafebad3c325bf4f" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 40857 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "10c286ffa08f437abafebad3c325bf4f" role: LEADER last_known_addr { host: "127.0.0.1" port: 40857 } } I1102 12:54:57.979814 32214 tablet_peer.cc:213] Configuration changed for tablet: 47353cc3163440698639e9c90f0176b1 in peer: 10c286ffa08f437abafebad3c325bf4f Changed from: seqno: -1 local: true peers { permanent_uuid: "10c286ffa08f437abafebad3c325bf4f" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 40857 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "10c286ffa08f437abafebad3c325bf4f" role: LEADER last_known_addr { host: "127.0.0.1" port: 40857 } } I1102 12:54:57.983893 32131 catalog_manager.cc:1127] Tablet: b4528c665a3149a9ac63fe038c159a71 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "10c286ffa08f437abafebad3c325bf4f" role: LEADER last_known_addr { host: "127.0.0.1" port: 40857 } } I1102 12:54:57.985716 32131 catalog_manager.cc:1127] Tablet: 47353cc3163440698639e9c90f0176b1 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "10c286ffa08f437abafebad3c325bf4f" role: LEADER last_known_addr { host: "127.0.0.1" port: 40857 } } I1102 12:54:57.994830 32136 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:44480: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:54:57.999974 32181 tablet_service.cc:264] Processing CreateTablet for tablet 43c8c0ffc82046109d55dfaff0ad4fb9 (table=client-testtb2 [id=fc9eb229980644d2bdcece6305149e5e]), range=["", ""] I1102 12:54:58.001019 32181 ts_tablet_manager.cc:460] Registered tablet 43c8c0ffc82046109d55dfaff0ad4fb9 I1102 12:54:58.001179 32211 ts_tablet_manager.cc:325] Bootstrapping tablet: 43c8c0ffc82046109d55dfaff0ad4fb9 I1102 12:54:58.001956 32211 tablet_bootstrap.cc:234] Tablet 43c8c0ffc82046109d55dfaff0ad4fb9: Bootstrap starting. I1102 12:54:58.005694 32211 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 43c8c0ffc82046109d55dfaff0ad4fb9 creating new one. I1102 12:54:58.006681 32211 tablet_bootstrap.cc:234] Tablet 43c8c0ffc82046109d55dfaff0ad4fb9: No bootstrap required, opened a new log I1102 12:54:58.006861 32211 ts_tablet_manager.cc:330] Time spent tablet 43c8c0ffc82046109d55dfaff0ad4fb9 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:54:58.007774 32211 ts_tablet_manager.cc:350] Time spent starting tablet 43c8c0ffc82046109d55dfaff0ad4fb9: real 0.001s user 0.000s sys 0.000s I1102 12:54:58.008009 32221 tablet_peer.cc:213] Configuration changed for tablet: 43c8c0ffc82046109d55dfaff0ad4fb9 in peer: 10c286ffa08f437abafebad3c325bf4f Changed from: seqno: -1 local: true peers { permanent_uuid: "10c286ffa08f437abafebad3c325bf4f" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 40857 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "10c286ffa08f437abafebad3c325bf4f" role: LEADER last_known_addr { host: "127.0.0.1" port: 40857 } } I1102 12:54:58.008625 32133 catalog_manager.cc:1127] Tablet: 43c8c0ffc82046109d55dfaff0ad4fb9 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "10c286ffa08f437abafebad3c325bf4f" role: LEADER last_known_addr { host: "127.0.0.1" port: 40857 } } I1102 12:54:58.180955 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:58.193435 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:54:58.193521 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:54:58.193584 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 43c8c0ffc82046109d55dfaff0ad4fb9 I1102 12:54:58.193642 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(43c8c0ffc82046109d55dfaff0ad4fb9) I1102 12:54:58.193658 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(43c8c0ffc82046109d55dfaff0ad4fb9) I1102 12:54:58.193675 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(43c8c0ffc82046109d55dfaff0ad4fb9) I1102 12:54:58.194141 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: b4528c665a3149a9ac63fe038c159a71 I1102 12:54:58.194169 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(b4528c665a3149a9ac63fe038c159a71) I1102 12:54:58.194182 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(b4528c665a3149a9ac63fe038c159a71) I1102 12:54:58.194201 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(b4528c665a3149a9ac63fe038c159a71) I1102 12:54:58.194625 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 47353cc3163440698639e9c90f0176b1 I1102 12:54:58.194654 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(47353cc3163440698639e9c90f0176b1) I1102 12:54:58.194665 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(47353cc3163440698639e9c90f0176b1) I1102 12:54:58.194679 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(47353cc3163440698639e9c90f0176b1) I1102 12:54:58.198148 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:58.198173 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:58.198207 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:58.199010 31819 master.cc:124] Master@127.0.0.1:36736 shutting down... I1102 12:54:58.210533 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:54:58.211011 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:54:58.211392 31819 master.cc:127] Master@127.0.0.1:36736 shutdown complete. [ OK ] ClientTest.TestScanAtSnapshot (434 ms) [ RUN ] ClientTest.TestScanMultiTablet I1102 12:54:58.362048 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanMultiTablet.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "9b65a2e9e4b048b0bc860ef7db50da2b" format_stamp: "Formatted at 2014-11-02 20:54:58 on qiwi" I1102 12:54:58.362149 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanMultiTablet.1414961696528764-31819/minicluster-data/master-0-root uuid: "9b65a2e9e4b048b0bc860ef7db50da2b" format_stamp: "Formatted at 2014-11-02 20:54:58 on qiwi" I1102 12:54:58.363004 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 492278 I1102 12:54:58.366165 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:58.366189 31819 webserver.cc:117] Document root disabled I1102 12:54:58.366395 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:54034 I1102 12:54:58.367630 32250 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:54:58.369534 32250 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:54:58.370380 32250 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:54:58.371664 32252 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 9b65a2e9e4b048b0bc860ef7db50da2b Changed from: seqno: 0 local: true peers { permanent_uuid: "9b65a2e9e4b048b0bc860ef7db50da2b" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "9b65a2e9e4b048b0bc860ef7db50da2b" role: LEADER } I1102 12:54:58.371726 32252 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "9b65a2e9e4b048b0bc860ef7db50da2b" role: LEADER } I1102 12:54:58.372197 32250 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:54:58.374028 32250 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:54:58.374893 32250 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:54:58.375936 32250 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:58.376231 32256 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 9b65a2e9e4b048b0bc860ef7db50da2b Changed from: seqno: 0 local: true peers { permanent_uuid: "9b65a2e9e4b048b0bc860ef7db50da2b" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "9b65a2e9e4b048b0bc860ef7db50da2b" role: LEADER } I1102 12:54:58.376286 32250 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:58.376289 32256 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "9b65a2e9e4b048b0bc860ef7db50da2b" role: LEADER } I1102 12:54:58.386940 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanMultiTablet.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "7a156512de624daf89ee3839554bcd33" format_stamp: "Formatted at 2014-11-02 20:54:58 on qiwi" I1102 12:54:58.387030 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanMultiTablet.1414961696528764-31819/minicluster-data/ts-0-root uuid: "7a156512de624daf89ee3839554bcd33" format_stamp: "Formatted at 2014-11-02 20:54:58 on qiwi" I1102 12:54:58.387876 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 492278 I1102 12:54:58.392235 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:58.392259 31819 webserver.cc:117] Document root disabled I1102 12:54:58.392483 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:56640 I1102 12:54:58.392622 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:54:58.394126 32299 heartbeater.cc:227] Registering TS with master... I1102 12:54:58.394188 32299 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:54:58.394505 32229 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "7a156512de624daf89ee3839554bcd33" instance_seqno: 1414961698391975 } with Master I1102 12:54:58.394968 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00216875s I1102 12:54:58.397830 32231 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:36041: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:54:58.403353 32277 tablet_service.cc:264] Processing CreateTablet for tablet 727ed51ab871491e8d103a5b0897740c (table=client-testtb [id=10235436e59843fb8fc770c549f62bf4]), range=["\x00\x00\x00\t", ""] I1102 12:54:58.403375 32278 tablet_service.cc:264] Processing CreateTablet for tablet 8813f19112ba444c86d2468d4ddb236e (table=client-testtb [id=10235436e59843fb8fc770c549f62bf4]), range=["", "\x00\x00\x00\t"] I1102 12:54:58.404270 32277 ts_tablet_manager.cc:460] Registered tablet 727ed51ab871491e8d103a5b0897740c I1102 12:54:58.404345 32278 ts_tablet_manager.cc:460] Registered tablet 8813f19112ba444c86d2468d4ddb236e I1102 12:54:58.404744 32308 ts_tablet_manager.cc:325] Bootstrapping tablet: 727ed51ab871491e8d103a5b0897740c I1102 12:54:58.404754 32309 ts_tablet_manager.cc:325] Bootstrapping tablet: 8813f19112ba444c86d2468d4ddb236e I1102 12:54:58.405213 32309 tablet_bootstrap.cc:234] Tablet 8813f19112ba444c86d2468d4ddb236e: Bootstrap starting. I1102 12:54:58.405215 32308 tablet_bootstrap.cc:234] Tablet 727ed51ab871491e8d103a5b0897740c: Bootstrap starting. I1102 12:54:58.409642 32308 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 727ed51ab871491e8d103a5b0897740c creating new one. I1102 12:54:58.409665 32309 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 8813f19112ba444c86d2468d4ddb236e creating new one. I1102 12:54:58.412147 32308 tablet_bootstrap.cc:234] Tablet 727ed51ab871491e8d103a5b0897740c: No bootstrap required, opened a new log I1102 12:54:58.412190 32309 tablet_bootstrap.cc:234] Tablet 8813f19112ba444c86d2468d4ddb236e: No bootstrap required, opened a new log I1102 12:54:58.412578 32308 ts_tablet_manager.cc:330] Time spent tablet 727ed51ab871491e8d103a5b0897740c bootstrap: real 0.008s user 0.005s sys 0.001s I1102 12:54:58.412607 32309 ts_tablet_manager.cc:330] Time spent tablet 8813f19112ba444c86d2468d4ddb236e bootstrap: real 0.008s user 0.005s sys 0.001s I1102 12:54:58.415187 32309 ts_tablet_manager.cc:350] Time spent starting tablet 8813f19112ba444c86d2468d4ddb236e: real 0.002s user 0.002s sys 0.001s I1102 12:54:58.415213 32308 ts_tablet_manager.cc:350] Time spent starting tablet 727ed51ab871491e8d103a5b0897740c: real 0.003s user 0.002s sys 0.001s I1102 12:54:58.415977 32312 tablet_peer.cc:213] Configuration changed for tablet: 727ed51ab871491e8d103a5b0897740c in peer: 7a156512de624daf89ee3839554bcd33 Changed from: seqno: -1 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 46158 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.416127 32313 tablet_peer.cc:213] Configuration changed for tablet: 8813f19112ba444c86d2468d4ddb236e in peer: 7a156512de624daf89ee3839554bcd33 Changed from: seqno: -1 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 46158 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.416882 32229 catalog_manager.cc:1127] Tablet: 727ed51ab871491e8d103a5b0897740c reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.418714 32229 catalog_manager.cc:1127] Tablet: 8813f19112ba444c86d2468d4ddb236e reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.423326 32233 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:36041: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:54:58.428774 32279 tablet_service.cc:264] Processing CreateTablet for tablet e05141c73a7a44ef9e1dbae739d1d6e4 (table=client-testtb2 [id=aed54d388bdd45aeb5a495792125c3d9]), range=["", ""] I1102 12:54:58.430068 32279 ts_tablet_manager.cc:460] Registered tablet e05141c73a7a44ef9e1dbae739d1d6e4 I1102 12:54:58.430204 32309 ts_tablet_manager.cc:325] Bootstrapping tablet: e05141c73a7a44ef9e1dbae739d1d6e4 I1102 12:54:58.430735 32309 tablet_bootstrap.cc:234] Tablet e05141c73a7a44ef9e1dbae739d1d6e4: Bootstrap starting. I1102 12:54:58.434303 32309 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: e05141c73a7a44ef9e1dbae739d1d6e4 creating new one. I1102 12:54:58.435637 32309 tablet_bootstrap.cc:234] Tablet e05141c73a7a44ef9e1dbae739d1d6e4: No bootstrap required, opened a new log I1102 12:54:58.435914 32309 ts_tablet_manager.cc:330] Time spent tablet e05141c73a7a44ef9e1dbae739d1d6e4 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:54:58.437221 32309 ts_tablet_manager.cc:350] Time spent starting tablet e05141c73a7a44ef9e1dbae739d1d6e4: real 0.001s user 0.001s sys 0.000s I1102 12:54:58.437481 32319 tablet_peer.cc:213] Configuration changed for tablet: e05141c73a7a44ef9e1dbae739d1d6e4 in peer: 7a156512de624daf89ee3839554bcd33 Changed from: seqno: -1 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 46158 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.445891 32232 catalog_manager.cc:1127] Tablet: e05141c73a7a44ef9e1dbae739d1d6e4 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.452591 32237 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:36041: name: "TestScanMultiTablet" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\n" pre_split_keys: "\000\000\000\024" pre_split_keys: "\000\000\000\036" pre_split_keys: "\000\000\000(" I1102 12:54:58.458461 32280 tablet_service.cc:264] Processing CreateTablet for tablet 93b884372f9e4035a9e69d90409725e0 (table=TestScanMultiTablet [id=f165c17ee56347a7a100fa6f5a33838f]), range=["", "\x00\x00\x00\n"] I1102 12:54:58.458549 32281 tablet_service.cc:264] Processing CreateTablet for tablet 86ebeb62252345f3baedf586ad5617cc (table=TestScanMultiTablet [id=f165c17ee56347a7a100fa6f5a33838f]), range=["\x00\x00\x00\n", "\x00\x00\x00\x14"] I1102 12:54:58.458652 32282 tablet_service.cc:264] Processing CreateTablet for tablet 3b3be7abf6504a3d938b79c464f25be6 (table=TestScanMultiTablet [id=f165c17ee56347a7a100fa6f5a33838f]), range=["\x00\x00\x00(", ""] I1102 12:54:58.458782 32283 tablet_service.cc:264] Processing CreateTablet for tablet af895e35eee24d0abce2a372d61d2af9 (table=TestScanMultiTablet [id=f165c17ee56347a7a100fa6f5a33838f]), range=["\x00\x00\x00\x1e", "\x00\x00\x00("] I1102 12:54:58.459470 32280 ts_tablet_manager.cc:460] Registered tablet 93b884372f9e4035a9e69d90409725e0 I1102 12:54:58.459485 32284 tablet_service.cc:264] Processing CreateTablet for tablet fbeb87e78eed45bd9be11a7c55dfe55e (table=TestScanMultiTablet [id=f165c17ee56347a7a100fa6f5a33838f]), range=["\x00\x00\x00\x14", "\x00\x00\x00\x1e"] I1102 12:54:58.459591 32308 ts_tablet_manager.cc:325] Bootstrapping tablet: 93b884372f9e4035a9e69d90409725e0 I1102 12:54:58.459751 32281 ts_tablet_manager.cc:460] Registered tablet 86ebeb62252345f3baedf586ad5617cc I1102 12:54:58.459825 32282 ts_tablet_manager.cc:460] Registered tablet 3b3be7abf6504a3d938b79c464f25be6 I1102 12:54:58.459899 32309 ts_tablet_manager.cc:325] Bootstrapping tablet: 86ebeb62252345f3baedf586ad5617cc I1102 12:54:58.460142 32283 ts_tablet_manager.cc:460] Registered tablet af895e35eee24d0abce2a372d61d2af9 I1102 12:54:58.460257 32308 tablet_bootstrap.cc:234] Tablet 93b884372f9e4035a9e69d90409725e0: Bootstrap starting. I1102 12:54:58.460927 32309 tablet_bootstrap.cc:234] Tablet 86ebeb62252345f3baedf586ad5617cc: Bootstrap starting. I1102 12:54:58.461009 32284 ts_tablet_manager.cc:460] Registered tablet fbeb87e78eed45bd9be11a7c55dfe55e I1102 12:54:58.461074 32321 ts_tablet_manager.cc:325] Bootstrapping tablet: 3b3be7abf6504a3d938b79c464f25be6 I1102 12:54:58.461118 32322 ts_tablet_manager.cc:325] Bootstrapping tablet: af895e35eee24d0abce2a372d61d2af9 I1102 12:54:58.461664 32323 ts_tablet_manager.cc:325] Bootstrapping tablet: fbeb87e78eed45bd9be11a7c55dfe55e I1102 12:54:58.462158 32321 tablet_bootstrap.cc:234] Tablet 3b3be7abf6504a3d938b79c464f25be6: Bootstrap starting. I1102 12:54:58.462496 32322 tablet_bootstrap.cc:234] Tablet af895e35eee24d0abce2a372d61d2af9: Bootstrap starting. I1102 12:54:58.463003 32323 tablet_bootstrap.cc:234] Tablet fbeb87e78eed45bd9be11a7c55dfe55e: Bootstrap starting. I1102 12:54:58.466359 32321 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 3b3be7abf6504a3d938b79c464f25be6 creating new one. I1102 12:54:58.467120 32308 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 93b884372f9e4035a9e69d90409725e0 creating new one. I1102 12:54:58.467625 32309 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 86ebeb62252345f3baedf586ad5617cc creating new one. I1102 12:54:58.468636 32321 tablet_bootstrap.cc:234] Tablet 3b3be7abf6504a3d938b79c464f25be6: No bootstrap required, opened a new log I1102 12:54:58.469092 32321 ts_tablet_manager.cc:330] Time spent tablet 3b3be7abf6504a3d938b79c464f25be6 bootstrap: real 0.008s user 0.004s sys 0.001s I1102 12:54:58.469281 32322 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: af895e35eee24d0abce2a372d61d2af9 creating new one. I1102 12:54:58.469501 32323 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: fbeb87e78eed45bd9be11a7c55dfe55e creating new one. I1102 12:54:58.470242 32309 tablet_bootstrap.cc:234] Tablet 86ebeb62252345f3baedf586ad5617cc: No bootstrap required, opened a new log I1102 12:54:58.470623 32308 tablet_bootstrap.cc:234] Tablet 93b884372f9e4035a9e69d90409725e0: No bootstrap required, opened a new log I1102 12:54:58.470693 32309 ts_tablet_manager.cc:330] Time spent tablet 86ebeb62252345f3baedf586ad5617cc bootstrap: real 0.011s user 0.006s sys 0.002s I1102 12:54:58.471206 32308 ts_tablet_manager.cc:330] Time spent tablet 93b884372f9e4035a9e69d90409725e0 bootstrap: real 0.011s user 0.006s sys 0.002s I1102 12:54:58.471864 32321 ts_tablet_manager.cc:350] Time spent starting tablet 3b3be7abf6504a3d938b79c464f25be6: real 0.003s user 0.001s sys 0.000s I1102 12:54:58.472425 32325 tablet_peer.cc:213] Configuration changed for tablet: 3b3be7abf6504a3d938b79c464f25be6 in peer: 7a156512de624daf89ee3839554bcd33 Changed from: seqno: -1 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 46158 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.473567 32308 ts_tablet_manager.cc:350] Time spent starting tablet 93b884372f9e4035a9e69d90409725e0: real 0.002s user 0.001s sys 0.000s I1102 12:54:58.473624 32323 tablet_bootstrap.cc:234] Tablet fbeb87e78eed45bd9be11a7c55dfe55e: No bootstrap required, opened a new log I1102 12:54:58.473629 32322 tablet_bootstrap.cc:234] Tablet af895e35eee24d0abce2a372d61d2af9: No bootstrap required, opened a new log I1102 12:54:58.473686 32309 ts_tablet_manager.cc:350] Time spent starting tablet 86ebeb62252345f3baedf586ad5617cc: real 0.003s user 0.001s sys 0.001s I1102 12:54:58.473887 32323 ts_tablet_manager.cc:330] Time spent tablet fbeb87e78eed45bd9be11a7c55dfe55e bootstrap: real 0.012s user 0.006s sys 0.002s I1102 12:54:58.473875 32329 tablet_peer.cc:213] Configuration changed for tablet: 86ebeb62252345f3baedf586ad5617cc in peer: 7a156512de624daf89ee3839554bcd33 Changed from: seqno: -1 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 46158 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.473950 32328 tablet_peer.cc:213] Configuration changed for tablet: 93b884372f9e4035a9e69d90409725e0 in peer: 7a156512de624daf89ee3839554bcd33 Changed from: seqno: -1 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 46158 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.474134 32322 ts_tablet_manager.cc:330] Time spent tablet af895e35eee24d0abce2a372d61d2af9 bootstrap: real 0.013s user 0.007s sys 0.002s I1102 12:54:58.475509 32323 ts_tablet_manager.cc:350] Time spent starting tablet fbeb87e78eed45bd9be11a7c55dfe55e: real 0.002s user 0.001s sys 0.000s I1102 12:54:58.475555 32322 ts_tablet_manager.cc:350] Time spent starting tablet af895e35eee24d0abce2a372d61d2af9: real 0.001s user 0.001s sys 0.000s I1102 12:54:58.475769 32335 tablet_peer.cc:213] Configuration changed for tablet: fbeb87e78eed45bd9be11a7c55dfe55e in peer: 7a156512de624daf89ee3839554bcd33 Changed from: seqno: -1 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 46158 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.475813 32336 tablet_peer.cc:213] Configuration changed for tablet: af895e35eee24d0abce2a372d61d2af9 in peer: 7a156512de624daf89ee3839554bcd33 Changed from: seqno: -1 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 46158 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.479704 32237 catalog_manager.cc:1127] Tablet: 93b884372f9e4035a9e69d90409725e0 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.481256 32237 catalog_manager.cc:1127] Tablet: 86ebeb62252345f3baedf586ad5617cc reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.482570 32237 catalog_manager.cc:1127] Tablet: 3b3be7abf6504a3d938b79c464f25be6 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.483958 32237 catalog_manager.cc:1127] Tablet: af895e35eee24d0abce2a372d61d2af9 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.485270 32237 catalog_manager.cc:1127] Tablet: fbeb87e78eed45bd9be11a7c55dfe55e reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "7a156512de624daf89ee3839554bcd33" role: LEADER last_known_addr { host: "127.0.0.1" port: 46158 } } I1102 12:54:58.643967 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:58.656385 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:54:58.656538 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:54:58.656652 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 93b884372f9e4035a9e69d90409725e0 I1102 12:54:58.656729 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(93b884372f9e4035a9e69d90409725e0) I1102 12:54:58.656754 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(93b884372f9e4035a9e69d90409725e0) I1102 12:54:58.656774 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(93b884372f9e4035a9e69d90409725e0) I1102 12:54:58.657245 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 727ed51ab871491e8d103a5b0897740c I1102 12:54:58.657274 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(727ed51ab871491e8d103a5b0897740c) I1102 12:54:58.657289 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(727ed51ab871491e8d103a5b0897740c) I1102 12:54:58.657305 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(727ed51ab871491e8d103a5b0897740c) I1102 12:54:58.657742 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 8813f19112ba444c86d2468d4ddb236e I1102 12:54:58.657770 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(8813f19112ba444c86d2468d4ddb236e) I1102 12:54:58.657783 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(8813f19112ba444c86d2468d4ddb236e) I1102 12:54:58.657799 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(8813f19112ba444c86d2468d4ddb236e) I1102 12:54:58.658221 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 86ebeb62252345f3baedf586ad5617cc I1102 12:54:58.658247 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(86ebeb62252345f3baedf586ad5617cc) I1102 12:54:58.658259 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(86ebeb62252345f3baedf586ad5617cc) I1102 12:54:58.658277 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(86ebeb62252345f3baedf586ad5617cc) I1102 12:54:58.658601 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 3b3be7abf6504a3d938b79c464f25be6 I1102 12:54:58.658625 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(3b3be7abf6504a3d938b79c464f25be6) I1102 12:54:58.658638 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(3b3be7abf6504a3d938b79c464f25be6) I1102 12:54:58.658653 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(3b3be7abf6504a3d938b79c464f25be6) I1102 12:54:58.658974 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: af895e35eee24d0abce2a372d61d2af9 I1102 12:54:58.659003 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(af895e35eee24d0abce2a372d61d2af9) I1102 12:54:58.659014 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(af895e35eee24d0abce2a372d61d2af9) I1102 12:54:58.659029 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(af895e35eee24d0abce2a372d61d2af9) I1102 12:54:58.659379 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: e05141c73a7a44ef9e1dbae739d1d6e4 I1102 12:54:58.659406 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(e05141c73a7a44ef9e1dbae739d1d6e4) I1102 12:54:58.659422 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(e05141c73a7a44ef9e1dbae739d1d6e4) I1102 12:54:58.659445 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(e05141c73a7a44ef9e1dbae739d1d6e4) I1102 12:54:58.659755 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: fbeb87e78eed45bd9be11a7c55dfe55e I1102 12:54:58.659781 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(fbeb87e78eed45bd9be11a7c55dfe55e) I1102 12:54:58.659793 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(fbeb87e78eed45bd9be11a7c55dfe55e) I1102 12:54:58.659807 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(fbeb87e78eed45bd9be11a7c55dfe55e) I1102 12:54:58.671800 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:58.671826 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:58.671856 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:58.673033 31819 master.cc:124] Master@127.0.0.1:44813 shutting down... I1102 12:54:58.685205 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:54:58.685886 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:54:58.686393 31819 master.cc:127] Master@127.0.0.1:44813 shutdown complete. [ OK ] ClientTest.TestScanMultiTablet (480 ms) [ RUN ] ClientTest.TestScanEmptyTable I1102 12:54:58.920372 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanEmptyTable.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "f90eedb89e07420f989bddee68bffa54" format_stamp: "Formatted at 2014-11-02 20:54:58 on qiwi" I1102 12:54:58.920773 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanEmptyTable.1414961696528764-31819/minicluster-data/master-0-root uuid: "f90eedb89e07420f989bddee68bffa54" format_stamp: "Formatted at 2014-11-02 20:54:58 on qiwi" I1102 12:54:58.923216 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 492278 I1102 12:54:58.927563 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:58.927582 31819 webserver.cc:117] Document root disabled I1102 12:54:58.927806 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:45361 I1102 12:54:58.929178 32369 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:54:58.931337 32369 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:54:58.932226 32369 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:54:58.933797 32371 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: f90eedb89e07420f989bddee68bffa54 Changed from: seqno: 0 local: true peers { permanent_uuid: "f90eedb89e07420f989bddee68bffa54" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "f90eedb89e07420f989bddee68bffa54" role: LEADER } I1102 12:54:58.933857 32371 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "f90eedb89e07420f989bddee68bffa54" role: LEADER } I1102 12:54:58.934260 32369 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:54:58.936380 32369 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:54:58.937245 32369 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:54:58.938312 32369 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:58.938652 32375 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: f90eedb89e07420f989bddee68bffa54 Changed from: seqno: 0 local: true peers { permanent_uuid: "f90eedb89e07420f989bddee68bffa54" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "f90eedb89e07420f989bddee68bffa54" role: LEADER } I1102 12:54:58.938709 32369 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:58.938709 32375 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "f90eedb89e07420f989bddee68bffa54" role: LEADER } I1102 12:54:58.951722 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanEmptyTable.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "fa8fd1e70c304aecb879bfea5178be28" format_stamp: "Formatted at 2014-11-02 20:54:58 on qiwi" I1102 12:54:58.952075 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanEmptyTable.1414961696528764-31819/minicluster-data/ts-0-root uuid: "fa8fd1e70c304aecb879bfea5178be28" format_stamp: "Formatted at 2014-11-02 20:54:58 on qiwi" I1102 12:54:58.953763 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 492278 I1102 12:54:58.960064 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:58.960086 31819 webserver.cc:117] Document root disabled I1102 12:54:58.960376 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:46631 I1102 12:54:58.960589 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:54:58.962462 32418 heartbeater.cc:227] Registering TS with master... I1102 12:54:58.962527 32418 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:54:58.963001 32348 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "fa8fd1e70c304aecb879bfea5178be28" instance_seqno: 1414961698959677 } with Master I1102 12:54:58.964046 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00325681s I1102 12:54:58.971509 32350 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:55055: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:54:58.983589 32396 tablet_service.cc:264] Processing CreateTablet for tablet 64b3e41a35574a4eafaa40ffcbaa8cf1 (table=client-testtb [id=ca7d233d32674940bd5e4e7351cdbd76]), range=["", "\x00\x00\x00\t"] I1102 12:54:58.983777 32397 tablet_service.cc:264] Processing CreateTablet for tablet 4f0cdd017dcf430698ff62253c7b9af1 (table=client-testtb [id=ca7d233d32674940bd5e4e7351cdbd76]), range=["\x00\x00\x00\t", ""] I1102 12:54:58.985692 32396 ts_tablet_manager.cc:460] Registered tablet 64b3e41a35574a4eafaa40ffcbaa8cf1 I1102 12:54:58.985888 32397 ts_tablet_manager.cc:460] Registered tablet 4f0cdd017dcf430698ff62253c7b9af1 I1102 12:54:58.986224 32427 ts_tablet_manager.cc:325] Bootstrapping tablet: 64b3e41a35574a4eafaa40ffcbaa8cf1 I1102 12:54:58.986714 32428 ts_tablet_manager.cc:325] Bootstrapping tablet: 4f0cdd017dcf430698ff62253c7b9af1 I1102 12:54:58.987352 32427 tablet_bootstrap.cc:234] Tablet 64b3e41a35574a4eafaa40ffcbaa8cf1: Bootstrap starting. I1102 12:54:58.987675 32428 tablet_bootstrap.cc:234] Tablet 4f0cdd017dcf430698ff62253c7b9af1: Bootstrap starting. I1102 12:54:58.992377 32427 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 64b3e41a35574a4eafaa40ffcbaa8cf1 creating new one. I1102 12:54:58.992472 32428 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 4f0cdd017dcf430698ff62253c7b9af1 creating new one. I1102 12:54:58.994107 32428 tablet_bootstrap.cc:234] Tablet 4f0cdd017dcf430698ff62253c7b9af1: No bootstrap required, opened a new log I1102 12:54:58.994143 32427 tablet_bootstrap.cc:234] Tablet 64b3e41a35574a4eafaa40ffcbaa8cf1: No bootstrap required, opened a new log I1102 12:54:58.994397 32428 ts_tablet_manager.cc:330] Time spent tablet 4f0cdd017dcf430698ff62253c7b9af1 bootstrap: real 0.008s user 0.006s sys 0.001s I1102 12:54:58.994407 32427 ts_tablet_manager.cc:330] Time spent tablet 64b3e41a35574a4eafaa40ffcbaa8cf1 bootstrap: real 0.008s user 0.006s sys 0.002s I1102 12:54:58.996059 32427 ts_tablet_manager.cc:350] Time spent starting tablet 64b3e41a35574a4eafaa40ffcbaa8cf1: real 0.002s user 0.001s sys 0.001s I1102 12:54:58.996095 32428 ts_tablet_manager.cc:350] Time spent starting tablet 4f0cdd017dcf430698ff62253c7b9af1: real 0.002s user 0.001s sys 0.000s I1102 12:54:58.996531 32432 tablet_peer.cc:213] Configuration changed for tablet: 64b3e41a35574a4eafaa40ffcbaa8cf1 in peer: fa8fd1e70c304aecb879bfea5178be28 Changed from: seqno: -1 local: true peers { permanent_uuid: "fa8fd1e70c304aecb879bfea5178be28" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 47791 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "fa8fd1e70c304aecb879bfea5178be28" role: LEADER last_known_addr { host: "127.0.0.1" port: 47791 } } I1102 12:54:58.996770 32431 tablet_peer.cc:213] Configuration changed for tablet: 4f0cdd017dcf430698ff62253c7b9af1 in peer: fa8fd1e70c304aecb879bfea5178be28 Changed from: seqno: -1 local: true peers { permanent_uuid: "fa8fd1e70c304aecb879bfea5178be28" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 47791 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "fa8fd1e70c304aecb879bfea5178be28" role: LEADER last_known_addr { host: "127.0.0.1" port: 47791 } } I1102 12:54:58.996852 32349 catalog_manager.cc:1127] Tablet: 64b3e41a35574a4eafaa40ffcbaa8cf1 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "fa8fd1e70c304aecb879bfea5178be28" role: LEADER last_known_addr { host: "127.0.0.1" port: 47791 } } I1102 12:54:58.998600 32349 catalog_manager.cc:1127] Tablet: 4f0cdd017dcf430698ff62253c7b9af1 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "fa8fd1e70c304aecb879bfea5178be28" role: LEADER last_known_addr { host: "127.0.0.1" port: 47791 } } I1102 12:54:59.007170 32354 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:55055: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:54:59.013056 32398 tablet_service.cc:264] Processing CreateTablet for tablet ee73e6c283cc43aca37d8ff3f7d46a92 (table=client-testtb2 [id=322b541dce1d4a5181cb431eee54d62a]), range=["", ""] I1102 12:54:59.014351 32398 ts_tablet_manager.cc:460] Registered tablet ee73e6c283cc43aca37d8ff3f7d46a92 I1102 12:54:59.014469 32427 ts_tablet_manager.cc:325] Bootstrapping tablet: ee73e6c283cc43aca37d8ff3f7d46a92 I1102 12:54:59.015210 32427 tablet_bootstrap.cc:234] Tablet ee73e6c283cc43aca37d8ff3f7d46a92: Bootstrap starting. I1102 12:54:59.018468 32427 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: ee73e6c283cc43aca37d8ff3f7d46a92 creating new one. I1102 12:54:59.020293 32427 tablet_bootstrap.cc:234] Tablet ee73e6c283cc43aca37d8ff3f7d46a92: No bootstrap required, opened a new log I1102 12:54:59.020730 32427 ts_tablet_manager.cc:330] Time spent tablet ee73e6c283cc43aca37d8ff3f7d46a92 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:54:59.022744 32427 ts_tablet_manager.cc:350] Time spent starting tablet ee73e6c283cc43aca37d8ff3f7d46a92: real 0.002s user 0.001s sys 0.000s I1102 12:54:59.023385 32438 tablet_peer.cc:213] Configuration changed for tablet: ee73e6c283cc43aca37d8ff3f7d46a92 in peer: fa8fd1e70c304aecb879bfea5178be28 Changed from: seqno: -1 local: true peers { permanent_uuid: "fa8fd1e70c304aecb879bfea5178be28" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 47791 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "fa8fd1e70c304aecb879bfea5178be28" role: LEADER last_known_addr { host: "127.0.0.1" port: 47791 } } I1102 12:54:59.026415 32352 catalog_manager.cc:1127] Tablet: ee73e6c283cc43aca37d8ff3f7d46a92 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "fa8fd1e70c304aecb879bfea5178be28" role: LEADER last_known_addr { host: "127.0.0.1" port: 47791 } } I1102 12:54:59.042310 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:59.059835 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:54:59.060021 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:54:59.060144 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 64b3e41a35574a4eafaa40ffcbaa8cf1 I1102 12:54:59.060256 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(64b3e41a35574a4eafaa40ffcbaa8cf1) I1102 12:54:59.060328 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(64b3e41a35574a4eafaa40ffcbaa8cf1) I1102 12:54:59.060382 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(64b3e41a35574a4eafaa40ffcbaa8cf1) I1102 12:54:59.061630 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: ee73e6c283cc43aca37d8ff3f7d46a92 I1102 12:54:59.061676 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(ee73e6c283cc43aca37d8ff3f7d46a92) I1102 12:54:59.061700 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(ee73e6c283cc43aca37d8ff3f7d46a92) I1102 12:54:59.061719 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(ee73e6c283cc43aca37d8ff3f7d46a92) I1102 12:54:59.062389 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 4f0cdd017dcf430698ff62253c7b9af1 I1102 12:54:59.062432 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(4f0cdd017dcf430698ff62253c7b9af1) I1102 12:54:59.062454 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(4f0cdd017dcf430698ff62253c7b9af1) I1102 12:54:59.062472 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(4f0cdd017dcf430698ff62253c7b9af1) I1102 12:54:59.069811 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:59.069839 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:59.069881 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:59.071331 31819 master.cc:124] Master@127.0.0.1:59832 shutting down... I1102 12:54:59.084511 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:54:59.085350 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:54:59.086069 31819 master.cc:127] Master@127.0.0.1:59832 shutdown complete. [ OK ] ClientTest.TestScanEmptyTable (399 ms) [ RUN ] ClientTest.TestScanEmptyProjection I1102 12:54:59.236696 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanEmptyProjection.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "cfa5b6a895ec435c8427e5aeb77114d7" format_stamp: "Formatted at 2014-11-02 20:54:59 on qiwi" I1102 12:54:59.236830 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanEmptyProjection.1414961696528764-31819/minicluster-data/master-0-root uuid: "cfa5b6a895ec435c8427e5aeb77114d7" format_stamp: "Formatted at 2014-11-02 20:54:59 on qiwi" I1102 12:54:59.237596 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 492778 I1102 12:54:59.240371 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:59.240386 31819 webserver.cc:117] Document root disabled I1102 12:54:59.240577 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:33693 I1102 12:54:59.241822 32467 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:54:59.243572 32467 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:54:59.244392 32467 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:54:59.245718 32469 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: cfa5b6a895ec435c8427e5aeb77114d7 Changed from: seqno: 0 local: true peers { permanent_uuid: "cfa5b6a895ec435c8427e5aeb77114d7" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "cfa5b6a895ec435c8427e5aeb77114d7" role: LEADER } I1102 12:54:59.245777 32469 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "cfa5b6a895ec435c8427e5aeb77114d7" role: LEADER } I1102 12:54:59.246162 32467 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:54:59.247897 32467 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:54:59.248746 32467 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:54:59.249757 32467 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:59.250047 32473 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: cfa5b6a895ec435c8427e5aeb77114d7 Changed from: seqno: 0 local: true peers { permanent_uuid: "cfa5b6a895ec435c8427e5aeb77114d7" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "cfa5b6a895ec435c8427e5aeb77114d7" role: LEADER } I1102 12:54:59.250094 32467 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:59.250097 32473 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "cfa5b6a895ec435c8427e5aeb77114d7" role: LEADER } I1102 12:54:59.260957 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanEmptyProjection.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "94095b7ce6f24d67829880be20fc1f0d" format_stamp: "Formatted at 2014-11-02 20:54:59 on qiwi" I1102 12:54:59.261050 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanEmptyProjection.1414961696528764-31819/minicluster-data/ts-0-root uuid: "94095b7ce6f24d67829880be20fc1f0d" format_stamp: "Formatted at 2014-11-02 20:54:59 on qiwi" I1102 12:54:59.261909 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 492778 I1102 12:54:59.266085 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:59.266101 31819 webserver.cc:117] Document root disabled I1102 12:54:59.266314 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:38081 I1102 12:54:59.266444 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:54:59.267889 32516 heartbeater.cc:227] Registering TS with master... I1102 12:54:59.267942 32516 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:54:59.268257 32446 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "94095b7ce6f24d67829880be20fc1f0d" instance_seqno: 1414961699265829 } with Master I1102 12:54:59.268748 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00218692s I1102 12:54:59.271417 32448 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:37564: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:54:59.276577 32494 tablet_service.cc:264] Processing CreateTablet for tablet 2b221b8b4e3e4fb3b068ba142803a66d (table=client-testtb [id=886d795ace0244a1b5f04bcb8d15cb3c]), range=["\x00\x00\x00\t", ""] I1102 12:54:59.276587 32495 tablet_service.cc:264] Processing CreateTablet for tablet 7f68ecdf4fd24c408cf75ba8312dbeea (table=client-testtb [id=886d795ace0244a1b5f04bcb8d15cb3c]), range=["", "\x00\x00\x00\t"] I1102 12:54:59.277472 32495 ts_tablet_manager.cc:460] Registered tablet 7f68ecdf4fd24c408cf75ba8312dbeea I1102 12:54:59.277554 32494 ts_tablet_manager.cc:460] Registered tablet 2b221b8b4e3e4fb3b068ba142803a66d I1102 12:54:59.277948 32525 ts_tablet_manager.cc:325] Bootstrapping tablet: 7f68ecdf4fd24c408cf75ba8312dbeea I1102 12:54:59.277957 32526 ts_tablet_manager.cc:325] Bootstrapping tablet: 2b221b8b4e3e4fb3b068ba142803a66d I1102 12:54:59.278321 32526 tablet_bootstrap.cc:234] Tablet 2b221b8b4e3e4fb3b068ba142803a66d: Bootstrap starting. I1102 12:54:59.278357 32525 tablet_bootstrap.cc:234] Tablet 7f68ecdf4fd24c408cf75ba8312dbeea: Bootstrap starting. I1102 12:54:59.280154 32526 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 2b221b8b4e3e4fb3b068ba142803a66d creating new one. I1102 12:54:59.280177 32525 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 7f68ecdf4fd24c408cf75ba8312dbeea creating new one. I1102 12:54:59.281236 32525 tablet_bootstrap.cc:234] Tablet 7f68ecdf4fd24c408cf75ba8312dbeea: No bootstrap required, opened a new log I1102 12:54:59.281388 32526 tablet_bootstrap.cc:234] Tablet 2b221b8b4e3e4fb3b068ba142803a66d: No bootstrap required, opened a new log I1102 12:54:59.281427 32525 ts_tablet_manager.cc:330] Time spent tablet 7f68ecdf4fd24c408cf75ba8312dbeea bootstrap: real 0.003s user 0.002s sys 0.000s I1102 12:54:59.281554 32526 ts_tablet_manager.cc:330] Time spent tablet 2b221b8b4e3e4fb3b068ba142803a66d bootstrap: real 0.004s user 0.002s sys 0.001s I1102 12:54:59.282446 32525 ts_tablet_manager.cc:350] Time spent starting tablet 7f68ecdf4fd24c408cf75ba8312dbeea: real 0.001s user 0.001s sys 0.000s I1102 12:54:59.282472 32526 ts_tablet_manager.cc:350] Time spent starting tablet 2b221b8b4e3e4fb3b068ba142803a66d: real 0.001s user 0.001s sys 0.000s I1102 12:54:59.283017 32530 tablet_peer.cc:213] Configuration changed for tablet: 2b221b8b4e3e4fb3b068ba142803a66d in peer: 94095b7ce6f24d67829880be20fc1f0d Changed from: seqno: -1 local: true peers { permanent_uuid: "94095b7ce6f24d67829880be20fc1f0d" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 55136 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "94095b7ce6f24d67829880be20fc1f0d" role: LEADER last_known_addr { host: "127.0.0.1" port: 55136 } } I1102 12:54:59.283017 32529 tablet_peer.cc:213] Configuration changed for tablet: 7f68ecdf4fd24c408cf75ba8312dbeea in peer: 94095b7ce6f24d67829880be20fc1f0d Changed from: seqno: -1 local: true peers { permanent_uuid: "94095b7ce6f24d67829880be20fc1f0d" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 55136 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "94095b7ce6f24d67829880be20fc1f0d" role: LEADER last_known_addr { host: "127.0.0.1" port: 55136 } } I1102 12:54:59.289657 32446 catalog_manager.cc:1127] Tablet: 2b221b8b4e3e4fb3b068ba142803a66d reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "94095b7ce6f24d67829880be20fc1f0d" role: LEADER last_known_addr { host: "127.0.0.1" port: 55136 } } I1102 12:54:59.291085 32446 catalog_manager.cc:1127] Tablet: 7f68ecdf4fd24c408cf75ba8312dbeea reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "94095b7ce6f24d67829880be20fc1f0d" role: LEADER last_known_addr { host: "127.0.0.1" port: 55136 } } I1102 12:54:59.298900 32451 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:37564: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:54:59.302749 32496 tablet_service.cc:264] Processing CreateTablet for tablet 89f9095d8d7549298fe878a5852dc2f7 (table=client-testtb2 [id=7b10cd517af24a08954019c1e890794b]), range=["", ""] I1102 12:54:59.303679 32496 ts_tablet_manager.cc:460] Registered tablet 89f9095d8d7549298fe878a5852dc2f7 I1102 12:54:59.303768 32525 ts_tablet_manager.cc:325] Bootstrapping tablet: 89f9095d8d7549298fe878a5852dc2f7 I1102 12:54:59.304131 32525 tablet_bootstrap.cc:234] Tablet 89f9095d8d7549298fe878a5852dc2f7: Bootstrap starting. I1102 12:54:59.306062 32525 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 89f9095d8d7549298fe878a5852dc2f7 creating new one. I1102 12:54:59.306917 32525 tablet_bootstrap.cc:234] Tablet 89f9095d8d7549298fe878a5852dc2f7: No bootstrap required, opened a new log I1102 12:54:59.307106 32525 ts_tablet_manager.cc:330] Time spent tablet 89f9095d8d7549298fe878a5852dc2f7 bootstrap: real 0.003s user 0.002s sys 0.001s I1102 12:54:59.308055 32525 ts_tablet_manager.cc:350] Time spent starting tablet 89f9095d8d7549298fe878a5852dc2f7: real 0.001s user 0.000s sys 0.000s I1102 12:54:59.308418 32536 tablet_peer.cc:213] Configuration changed for tablet: 89f9095d8d7549298fe878a5852dc2f7 in peer: 94095b7ce6f24d67829880be20fc1f0d Changed from: seqno: -1 local: true peers { permanent_uuid: "94095b7ce6f24d67829880be20fc1f0d" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 55136 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "94095b7ce6f24d67829880be20fc1f0d" role: LEADER last_known_addr { host: "127.0.0.1" port: 55136 } } I1102 12:54:59.313645 32448 catalog_manager.cc:1127] Tablet: 89f9095d8d7549298fe878a5852dc2f7 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "94095b7ce6f24d67829880be20fc1f0d" role: LEADER last_known_addr { host: "127.0.0.1" port: 55136 } } I1102 12:54:59.477874 31819 client-test.cc:651] Time spent Scanning with no projected columns: real 0.007s user 0.001s sys 0.000s I1102 12:54:59.477932 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:59.490321 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:54:59.490378 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:54:59.490417 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 2b221b8b4e3e4fb3b068ba142803a66d I1102 12:54:59.490443 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(2b221b8b4e3e4fb3b068ba142803a66d) I1102 12:54:59.490464 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(2b221b8b4e3e4fb3b068ba142803a66d) I1102 12:54:59.490476 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(2b221b8b4e3e4fb3b068ba142803a66d) I1102 12:54:59.490936 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 7f68ecdf4fd24c408cf75ba8312dbeea I1102 12:54:59.490957 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(7f68ecdf4fd24c408cf75ba8312dbeea) I1102 12:54:59.490968 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(7f68ecdf4fd24c408cf75ba8312dbeea) I1102 12:54:59.490977 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(7f68ecdf4fd24c408cf75ba8312dbeea) I1102 12:54:59.491329 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 89f9095d8d7549298fe878a5852dc2f7 I1102 12:54:59.491348 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(89f9095d8d7549298fe878a5852dc2f7) I1102 12:54:59.491358 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(89f9095d8d7549298fe878a5852dc2f7) I1102 12:54:59.491370 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(89f9095d8d7549298fe878a5852dc2f7) I1102 12:54:59.494858 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:59.494874 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:59.494897 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:59.495599 31819 master.cc:124] Master@127.0.0.1:56579 shutting down... I1102 12:54:59.507175 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:54:59.507612 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:54:59.507994 31819 master.cc:127] Master@127.0.0.1:56579 shutdown complete. [ OK ] ClientTest.TestScanEmptyProjection (422 ms) [ RUN ] ClientTest.TestScanPredicateKeyColNotProjected I1102 12:54:59.651963 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanPredicateKeyColNotProjected.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "a39a7c18558c4ad2ac65998fe7b5c817" format_stamp: "Formatted at 2014-11-02 20:54:59 on qiwi" I1102 12:54:59.652084 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanPredicateKeyColNotProjected.1414961696528764-31819/minicluster-data/master-0-root uuid: "a39a7c18558c4ad2ac65998fe7b5c817" format_stamp: "Formatted at 2014-11-02 20:54:59 on qiwi" I1102 12:54:59.652798 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 492778 I1102 12:54:59.655625 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:59.655640 31819 webserver.cc:117] Document root disabled I1102 12:54:59.655830 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:34061 I1102 12:54:59.657086 32574 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:54:59.658942 32574 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:54:59.659749 32574 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:54:59.661136 32576 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: a39a7c18558c4ad2ac65998fe7b5c817 Changed from: seqno: 0 local: true peers { permanent_uuid: "a39a7c18558c4ad2ac65998fe7b5c817" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "a39a7c18558c4ad2ac65998fe7b5c817" role: LEADER } I1102 12:54:59.661193 32576 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "a39a7c18558c4ad2ac65998fe7b5c817" role: LEADER } I1102 12:54:59.661584 32574 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:54:59.663380 32574 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:54:59.664170 32574 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:54:59.665233 32574 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:59.665606 32580 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: a39a7c18558c4ad2ac65998fe7b5c817 Changed from: seqno: 0 local: true peers { permanent_uuid: "a39a7c18558c4ad2ac65998fe7b5c817" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "a39a7c18558c4ad2ac65998fe7b5c817" role: LEADER } I1102 12:54:59.665655 32574 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:54:59.665657 32580 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "a39a7c18558c4ad2ac65998fe7b5c817" role: LEADER } I1102 12:54:59.676560 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanPredicateKeyColNotProjected.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "9fe08d940d74470bbb357e11bd47698b" format_stamp: "Formatted at 2014-11-02 20:54:59 on qiwi" I1102 12:54:59.676651 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanPredicateKeyColNotProjected.1414961696528764-31819/minicluster-data/ts-0-root uuid: "9fe08d940d74470bbb357e11bd47698b" format_stamp: "Formatted at 2014-11-02 20:54:59 on qiwi" I1102 12:54:59.677500 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 492778 I1102 12:54:59.682359 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:54:59.682379 31819 webserver.cc:117] Document root disabled I1102 12:54:59.682612 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:46946 I1102 12:54:59.682795 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:54:59.684322 32623 heartbeater.cc:227] Registering TS with master... I1102 12:54:59.684375 32623 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:54:59.684680 32553 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "9fe08d940d74470bbb357e11bd47698b" instance_seqno: 1414961699682094 } with Master I1102 12:54:59.685097 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00219103s I1102 12:54:59.688251 32555 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:58693: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:54:59.694571 32601 tablet_service.cc:264] Processing CreateTablet for tablet cae8fe0b24634020bd6db4e28ab34609 (table=client-testtb [id=8077fee17e704ca1a0bca15bcb2d25f0]), range=["\x00\x00\x00\t", ""] I1102 12:54:59.694787 32602 tablet_service.cc:264] Processing CreateTablet for tablet fca0679ad30949ca8fbff494bcd02105 (table=client-testtb [id=8077fee17e704ca1a0bca15bcb2d25f0]), range=["", "\x00\x00\x00\t"] I1102 12:54:59.695797 32601 ts_tablet_manager.cc:460] Registered tablet cae8fe0b24634020bd6db4e28ab34609 I1102 12:54:59.695849 32602 ts_tablet_manager.cc:460] Registered tablet fca0679ad30949ca8fbff494bcd02105 I1102 12:54:59.696287 32632 ts_tablet_manager.cc:325] Bootstrapping tablet: cae8fe0b24634020bd6db4e28ab34609 I1102 12:54:59.696308 32633 ts_tablet_manager.cc:325] Bootstrapping tablet: fca0679ad30949ca8fbff494bcd02105 I1102 12:54:59.696766 32632 tablet_bootstrap.cc:234] Tablet cae8fe0b24634020bd6db4e28ab34609: Bootstrap starting. I1102 12:54:59.696779 32633 tablet_bootstrap.cc:234] Tablet fca0679ad30949ca8fbff494bcd02105: Bootstrap starting. I1102 12:54:59.702200 32632 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: cae8fe0b24634020bd6db4e28ab34609 creating new one. I1102 12:54:59.702510 32633 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: fca0679ad30949ca8fbff494bcd02105 creating new one. I1102 12:54:59.704530 32632 tablet_bootstrap.cc:234] Tablet cae8fe0b24634020bd6db4e28ab34609: No bootstrap required, opened a new log I1102 12:54:59.705093 32632 ts_tablet_manager.cc:330] Time spent tablet cae8fe0b24634020bd6db4e28ab34609 bootstrap: real 0.009s user 0.006s sys 0.002s I1102 12:54:59.705173 32633 tablet_bootstrap.cc:234] Tablet fca0679ad30949ca8fbff494bcd02105: No bootstrap required, opened a new log I1102 12:54:59.705711 32633 ts_tablet_manager.cc:330] Time spent tablet fca0679ad30949ca8fbff494bcd02105 bootstrap: real 0.009s user 0.006s sys 0.002s I1102 12:54:59.707427 32632 ts_tablet_manager.cc:350] Time spent starting tablet cae8fe0b24634020bd6db4e28ab34609: real 0.002s user 0.001s sys 0.000s I1102 12:54:59.708444 32633 ts_tablet_manager.cc:350] Time spent starting tablet fca0679ad30949ca8fbff494bcd02105: real 0.003s user 0.001s sys 0.001s I1102 12:54:59.708400 32636 tablet_peer.cc:213] Configuration changed for tablet: cae8fe0b24634020bd6db4e28ab34609 in peer: 9fe08d940d74470bbb357e11bd47698b Changed from: seqno: -1 local: true peers { permanent_uuid: "9fe08d940d74470bbb357e11bd47698b" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 59356 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "9fe08d940d74470bbb357e11bd47698b" role: LEADER last_known_addr { host: "127.0.0.1" port: 59356 } } I1102 12:54:59.708976 32637 tablet_peer.cc:213] Configuration changed for tablet: fca0679ad30949ca8fbff494bcd02105 in peer: 9fe08d940d74470bbb357e11bd47698b Changed from: seqno: -1 local: true peers { permanent_uuid: "9fe08d940d74470bbb357e11bd47698b" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 59356 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "9fe08d940d74470bbb357e11bd47698b" role: LEADER last_known_addr { host: "127.0.0.1" port: 59356 } } I1102 12:54:59.718706 32556 catalog_manager.cc:1127] Tablet: cae8fe0b24634020bd6db4e28ab34609 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "9fe08d940d74470bbb357e11bd47698b" role: LEADER last_known_addr { host: "127.0.0.1" port: 59356 } } I1102 12:54:59.723541 32556 catalog_manager.cc:1127] Tablet: fca0679ad30949ca8fbff494bcd02105 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "9fe08d940d74470bbb357e11bd47698b" role: LEADER last_known_addr { host: "127.0.0.1" port: 59356 } } I1102 12:54:59.741040 32552 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:58693: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:54:59.751315 32603 tablet_service.cc:264] Processing CreateTablet for tablet 0cc06d5da10a4179a8fa2eca342666f1 (table=client-testtb2 [id=a417ca25d24249928703e16be9ec6d4c]), range=["", ""] I1102 12:54:59.753897 32603 ts_tablet_manager.cc:460] Registered tablet 0cc06d5da10a4179a8fa2eca342666f1 I1102 12:54:59.754113 32632 ts_tablet_manager.cc:325] Bootstrapping tablet: 0cc06d5da10a4179a8fa2eca342666f1 I1102 12:54:59.755195 32632 tablet_bootstrap.cc:234] Tablet 0cc06d5da10a4179a8fa2eca342666f1: Bootstrap starting. I1102 12:54:59.759456 32632 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 0cc06d5da10a4179a8fa2eca342666f1 creating new one. I1102 12:54:59.761147 32632 tablet_bootstrap.cc:234] Tablet 0cc06d5da10a4179a8fa2eca342666f1: No bootstrap required, opened a new log I1102 12:54:59.761544 32632 ts_tablet_manager.cc:330] Time spent tablet 0cc06d5da10a4179a8fa2eca342666f1 bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:54:59.763219 32632 ts_tablet_manager.cc:350] Time spent starting tablet 0cc06d5da10a4179a8fa2eca342666f1: real 0.002s user 0.001s sys 0.000s I1102 12:54:59.763761 32642 tablet_peer.cc:213] Configuration changed for tablet: 0cc06d5da10a4179a8fa2eca342666f1 in peer: 9fe08d940d74470bbb357e11bd47698b Changed from: seqno: -1 local: true peers { permanent_uuid: "9fe08d940d74470bbb357e11bd47698b" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 59356 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "9fe08d940d74470bbb357e11bd47698b" role: LEADER last_known_addr { host: "127.0.0.1" port: 59356 } } I1102 12:54:59.774189 32552 catalog_manager.cc:1127] Tablet: 0cc06d5da10a4179a8fa2eca342666f1 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "9fe08d940d74470bbb357e11bd47698b" role: LEADER last_known_addr { host: "127.0.0.1" port: 59356 } } I1102 12:54:59.957857 31819 client-test.cc:683] Time spent Scanning with predicate columns not projected: real 0.010s user 0.001s sys 0.000s I1102 12:54:59.957983 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:59.960911 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:54:59.961228 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:54:59.961298 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 0cc06d5da10a4179a8fa2eca342666f1 I1102 12:54:59.961341 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(0cc06d5da10a4179a8fa2eca342666f1) I1102 12:54:59.961356 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(0cc06d5da10a4179a8fa2eca342666f1) I1102 12:54:59.961374 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(0cc06d5da10a4179a8fa2eca342666f1) I1102 12:54:59.962033 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: cae8fe0b24634020bd6db4e28ab34609 I1102 12:54:59.962069 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(cae8fe0b24634020bd6db4e28ab34609) I1102 12:54:59.962090 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(cae8fe0b24634020bd6db4e28ab34609) I1102 12:54:59.962110 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(cae8fe0b24634020bd6db4e28ab34609) I1102 12:54:59.962561 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: fca0679ad30949ca8fbff494bcd02105 I1102 12:54:59.962599 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(fca0679ad30949ca8fbff494bcd02105) I1102 12:54:59.962617 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(fca0679ad30949ca8fbff494bcd02105) I1102 12:54:59.962637 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(fca0679ad30949ca8fbff494bcd02105) I1102 12:54:59.968003 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:59.968049 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:54:59.968078 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:54:59.969370 31819 master.cc:124] Master@127.0.0.1:55672 shutting down... I1102 12:54:59.981123 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:54:59.982136 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:54:59.982527 31819 master.cc:127] Master@127.0.0.1:55672 shutdown complete. [ OK ] ClientTest.TestScanPredicateKeyColNotProjected (474 ms) [ RUN ] ClientTest.TestScanPredicateNonKeyColNotProjected I1102 12:55:00.138046 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanPredicateNonKeyColNotProjected.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "34c86ec7c3f84f10b72ed45c49148930" format_stamp: "Formatted at 2014-11-02 20:54:59 on qiwi" I1102 12:55:00.138180 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanPredicateNonKeyColNotProjected.1414961696528764-31819/minicluster-data/master-0-root uuid: "34c86ec7c3f84f10b72ed45c49148930" format_stamp: "Formatted at 2014-11-02 20:54:59 on qiwi" I1102 12:55:00.139049 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 493278 I1102 12:55:00.142254 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:00.142282 31819 webserver.cc:117] Document root disabled I1102 12:55:00.142514 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:37862 I1102 12:55:00.143754 32672 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:00.145938 32672 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:00.146829 32672 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:00.148217 32674 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 34c86ec7c3f84f10b72ed45c49148930 Changed from: seqno: 0 local: true peers { permanent_uuid: "34c86ec7c3f84f10b72ed45c49148930" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "34c86ec7c3f84f10b72ed45c49148930" role: LEADER } I1102 12:55:00.148278 32674 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "34c86ec7c3f84f10b72ed45c49148930" role: LEADER } I1102 12:55:00.148828 32672 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:00.150816 32672 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:00.151713 32672 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:00.152730 32672 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:00.153026 32678 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 34c86ec7c3f84f10b72ed45c49148930 Changed from: seqno: 0 local: true peers { permanent_uuid: "34c86ec7c3f84f10b72ed45c49148930" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "34c86ec7c3f84f10b72ed45c49148930" role: LEADER } I1102 12:55:00.153085 32672 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:00.153090 32678 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "34c86ec7c3f84f10b72ed45c49148930" role: LEADER } I1102 12:55:00.164227 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanPredicateNonKeyColNotProjected.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "09b8d9e042404fd18e0f369924cd62d8" format_stamp: "Formatted at 2014-11-02 20:55:00 on qiwi" I1102 12:55:00.164327 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanPredicateNonKeyColNotProjected.1414961696528764-31819/minicluster-data/ts-0-root uuid: "09b8d9e042404fd18e0f369924cd62d8" format_stamp: "Formatted at 2014-11-02 20:55:00 on qiwi" I1102 12:55:00.165199 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 493278 I1102 12:55:00.169549 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:00.169575 31819 webserver.cc:117] Document root disabled I1102 12:55:00.169795 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:37816 I1102 12:55:00.169925 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:00.171355 32721 heartbeater.cc:227] Registering TS with master... I1102 12:55:00.171422 32721 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:00.171743 32651 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "09b8d9e042404fd18e0f369924cd62d8" instance_seqno: 1414961700169285 } with Master I1102 12:55:00.172261 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00216293s I1102 12:55:00.175212 32653 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:47815: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:00.180896 32699 tablet_service.cc:264] Processing CreateTablet for tablet 4b692d61696849b0b75a8f32819e5467 (table=client-testtb [id=fbf856c4403e4690a6a9cf97b59ced8a]), range=["", "\x00\x00\x00\t"] I1102 12:55:00.180948 32700 tablet_service.cc:264] Processing CreateTablet for tablet 4fe0719417a14383a3d416f5eb44a281 (table=client-testtb [id=fbf856c4403e4690a6a9cf97b59ced8a]), range=["\x00\x00\x00\t", ""] I1102 12:55:00.181803 32699 ts_tablet_manager.cc:460] Registered tablet 4b692d61696849b0b75a8f32819e5467 I1102 12:55:00.181826 32700 ts_tablet_manager.cc:460] Registered tablet 4fe0719417a14383a3d416f5eb44a281 I1102 12:55:00.182322 32731 ts_tablet_manager.cc:325] Bootstrapping tablet: 4fe0719417a14383a3d416f5eb44a281 I1102 12:55:00.182322 32730 ts_tablet_manager.cc:325] Bootstrapping tablet: 4b692d61696849b0b75a8f32819e5467 I1102 12:55:00.183073 32730 tablet_bootstrap.cc:234] Tablet 4b692d61696849b0b75a8f32819e5467: Bootstrap starting. I1102 12:55:00.183073 32731 tablet_bootstrap.cc:234] Tablet 4fe0719417a14383a3d416f5eb44a281: Bootstrap starting. I1102 12:55:00.186525 32731 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 4fe0719417a14383a3d416f5eb44a281 creating new one. I1102 12:55:00.186712 32730 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 4b692d61696849b0b75a8f32819e5467 creating new one. I1102 12:55:00.187919 32731 tablet_bootstrap.cc:234] Tablet 4fe0719417a14383a3d416f5eb44a281: No bootstrap required, opened a new log I1102 12:55:00.188149 32731 ts_tablet_manager.cc:330] Time spent tablet 4fe0719417a14383a3d416f5eb44a281 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:00.188632 32730 tablet_bootstrap.cc:234] Tablet 4b692d61696849b0b75a8f32819e5467: No bootstrap required, opened a new log I1102 12:55:00.188817 32730 ts_tablet_manager.cc:330] Time spent tablet 4b692d61696849b0b75a8f32819e5467 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:00.189282 32731 ts_tablet_manager.cc:350] Time spent starting tablet 4fe0719417a14383a3d416f5eb44a281: real 0.001s user 0.001s sys 0.000s I1102 12:55:00.189702 32730 ts_tablet_manager.cc:350] Time spent starting tablet 4b692d61696849b0b75a8f32819e5467: real 0.001s user 0.000s sys 0.000s I1102 12:55:00.189646 32735 tablet_peer.cc:213] Configuration changed for tablet: 4fe0719417a14383a3d416f5eb44a281 in peer: 09b8d9e042404fd18e0f369924cd62d8 Changed from: seqno: -1 local: true peers { permanent_uuid: "09b8d9e042404fd18e0f369924cd62d8" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 47765 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "09b8d9e042404fd18e0f369924cd62d8" role: LEADER last_known_addr { host: "127.0.0.1" port: 47765 } } I1102 12:55:00.189898 32734 tablet_peer.cc:213] Configuration changed for tablet: 4b692d61696849b0b75a8f32819e5467 in peer: 09b8d9e042404fd18e0f369924cd62d8 Changed from: seqno: -1 local: true peers { permanent_uuid: "09b8d9e042404fd18e0f369924cd62d8" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 47765 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "09b8d9e042404fd18e0f369924cd62d8" role: LEADER last_known_addr { host: "127.0.0.1" port: 47765 } } I1102 12:55:00.193248 32651 catalog_manager.cc:1127] Tablet: 4b692d61696849b0b75a8f32819e5467 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "09b8d9e042404fd18e0f369924cd62d8" role: LEADER last_known_addr { host: "127.0.0.1" port: 47765 } } I1102 12:55:00.194715 32651 catalog_manager.cc:1127] Tablet: 4fe0719417a14383a3d416f5eb44a281 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "09b8d9e042404fd18e0f369924cd62d8" role: LEADER last_known_addr { host: "127.0.0.1" port: 47765 } } I1102 12:55:00.197839 32655 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:47815: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:00.201679 32701 tablet_service.cc:264] Processing CreateTablet for tablet b0831cb5e19d4d8fa6ed937a1f3553b7 (table=client-testtb2 [id=e7859a8cbea345b998868ca7272f3f95]), range=["", ""] I1102 12:55:00.202525 32701 ts_tablet_manager.cc:460] Registered tablet b0831cb5e19d4d8fa6ed937a1f3553b7 I1102 12:55:00.202600 32731 ts_tablet_manager.cc:325] Bootstrapping tablet: b0831cb5e19d4d8fa6ed937a1f3553b7 I1102 12:55:00.202999 32731 tablet_bootstrap.cc:234] Tablet b0831cb5e19d4d8fa6ed937a1f3553b7: Bootstrap starting. I1102 12:55:00.204974 32731 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: b0831cb5e19d4d8fa6ed937a1f3553b7 creating new one. I1102 12:55:00.205914 32731 tablet_bootstrap.cc:234] Tablet b0831cb5e19d4d8fa6ed937a1f3553b7: No bootstrap required, opened a new log I1102 12:55:00.206079 32731 ts_tablet_manager.cc:330] Time spent tablet b0831cb5e19d4d8fa6ed937a1f3553b7 bootstrap: real 0.003s user 0.002s sys 0.001s I1102 12:55:00.207064 32731 ts_tablet_manager.cc:350] Time spent starting tablet b0831cb5e19d4d8fa6ed937a1f3553b7: real 0.001s user 0.000s sys 0.000s I1102 12:55:00.207308 32740 tablet_peer.cc:213] Configuration changed for tablet: b0831cb5e19d4d8fa6ed937a1f3553b7 in peer: 09b8d9e042404fd18e0f369924cd62d8 Changed from: seqno: -1 local: true peers { permanent_uuid: "09b8d9e042404fd18e0f369924cd62d8" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 47765 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "09b8d9e042404fd18e0f369924cd62d8" role: LEADER last_known_addr { host: "127.0.0.1" port: 47765 } } I1102 12:55:00.217319 32653 catalog_manager.cc:1127] Tablet: b0831cb5e19d4d8fa6ed937a1f3553b7 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "09b8d9e042404fd18e0f369924cd62d8" role: LEADER last_known_addr { host: "127.0.0.1" port: 47765 } } I1102 12:55:00.375543 31819 client-test.cc:722] Time spent Scanning with predicate columns not projected: real 0.009s user 0.001s sys 0.000s I1102 12:55:00.375617 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:00.388411 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:00.388475 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:00.388547 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: b0831cb5e19d4d8fa6ed937a1f3553b7 I1102 12:55:00.388578 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(b0831cb5e19d4d8fa6ed937a1f3553b7) I1102 12:55:00.388592 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(b0831cb5e19d4d8fa6ed937a1f3553b7) I1102 12:55:00.388600 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(b0831cb5e19d4d8fa6ed937a1f3553b7) I1102 12:55:00.389142 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 4b692d61696849b0b75a8f32819e5467 I1102 12:55:00.389161 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(4b692d61696849b0b75a8f32819e5467) I1102 12:55:00.389171 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(4b692d61696849b0b75a8f32819e5467) I1102 12:55:00.389255 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(4b692d61696849b0b75a8f32819e5467) I1102 12:55:00.389859 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 4fe0719417a14383a3d416f5eb44a281 I1102 12:55:00.389879 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(4fe0719417a14383a3d416f5eb44a281) I1102 12:55:00.389894 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(4fe0719417a14383a3d416f5eb44a281) I1102 12:55:00.389905 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(4fe0719417a14383a3d416f5eb44a281) I1102 12:55:00.393780 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:00.393801 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:00.393828 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:00.394587 31819 master.cc:124] Master@127.0.0.1:47725 shutting down... I1102 12:55:00.406162 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:00.406637 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:00.407011 31819 master.cc:127] Master@127.0.0.1:47725 shutdown complete. [ OK ] ClientTest.TestScanPredicateNonKeyColNotProjected (426 ms) [ RUN ] ClientTest.TestCloseScanner I1102 12:55:00.561794 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestCloseScanner.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "3208884161ed4f8687e5aa121f22c5dc" format_stamp: "Formatted at 2014-11-02 20:55:00 on qiwi" I1102 12:55:00.561926 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestCloseScanner.1414961696528764-31819/minicluster-data/master-0-root uuid: "3208884161ed4f8687e5aa121f22c5dc" format_stamp: "Formatted at 2014-11-02 20:55:00 on qiwi" I1102 12:55:00.562732 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 493278 I1102 12:55:00.565675 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:00.565691 31819 webserver.cc:117] Document root disabled I1102 12:55:00.565901 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:34521 I1102 12:55:00.567157 301 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:00.568966 301 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:00.569696 301 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:00.571125 303 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 3208884161ed4f8687e5aa121f22c5dc Changed from: seqno: 0 local: true peers { permanent_uuid: "3208884161ed4f8687e5aa121f22c5dc" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "3208884161ed4f8687e5aa121f22c5dc" role: LEADER } I1102 12:55:00.571202 303 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "3208884161ed4f8687e5aa121f22c5dc" role: LEADER } I1102 12:55:00.572007 301 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:00.573891 301 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:00.574676 301 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:00.575688 301 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:00.576086 307 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 3208884161ed4f8687e5aa121f22c5dc Changed from: seqno: 0 local: true peers { permanent_uuid: "3208884161ed4f8687e5aa121f22c5dc" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "3208884161ed4f8687e5aa121f22c5dc" role: LEADER } I1102 12:55:00.576133 301 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:00.576139 307 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "3208884161ed4f8687e5aa121f22c5dc" role: LEADER } I1102 12:55:00.587131 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestCloseScanner.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "f47b8dfad3cc41de8cf483a28bbc4cf7" format_stamp: "Formatted at 2014-11-02 20:55:00 on qiwi" I1102 12:55:00.587239 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestCloseScanner.1414961696528764-31819/minicluster-data/ts-0-root uuid: "f47b8dfad3cc41de8cf483a28bbc4cf7" format_stamp: "Formatted at 2014-11-02 20:55:00 on qiwi" I1102 12:55:00.587936 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 493278 I1102 12:55:00.591893 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:00.591912 31819 webserver.cc:117] Document root disabled I1102 12:55:00.592133 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:55452 I1102 12:55:00.592258 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:00.595329 352 heartbeater.cc:227] Registering TS with master... I1102 12:55:00.595517 352 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:00.596752 32748 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "f47b8dfad3cc41de8cf483a28bbc4cf7" instance_seqno: 1414961700591635 } with Master I1102 12:55:00.596923 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00456609s I1102 12:55:00.602530 32750 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:35271: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:00.612812 330 tablet_service.cc:264] Processing CreateTablet for tablet d2150c249b9643698fb987d85678a3e4 (table=client-testtb [id=01ccff4eeae748bf82bd8822cdf3ebbd]), range=["", "\x00\x00\x00\t"] I1102 12:55:00.612892 331 tablet_service.cc:264] Processing CreateTablet for tablet 1e197352e50a4c37ad9ac16503d7dd58 (table=client-testtb [id=01ccff4eeae748bf82bd8822cdf3ebbd]), range=["\x00\x00\x00\t", ""] I1102 12:55:00.614481 331 ts_tablet_manager.cc:460] Registered tablet 1e197352e50a4c37ad9ac16503d7dd58 I1102 12:55:00.614521 330 ts_tablet_manager.cc:460] Registered tablet d2150c249b9643698fb987d85678a3e4 I1102 12:55:00.614843 362 ts_tablet_manager.cc:325] Bootstrapping tablet: 1e197352e50a4c37ad9ac16503d7dd58 I1102 12:55:00.615258 365 ts_tablet_manager.cc:325] Bootstrapping tablet: d2150c249b9643698fb987d85678a3e4 I1102 12:55:00.615566 362 tablet_bootstrap.cc:234] Tablet 1e197352e50a4c37ad9ac16503d7dd58: Bootstrap starting. I1102 12:55:00.615989 365 tablet_bootstrap.cc:234] Tablet d2150c249b9643698fb987d85678a3e4: Bootstrap starting. I1102 12:55:00.619674 362 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 1e197352e50a4c37ad9ac16503d7dd58 creating new one. I1102 12:55:00.620700 365 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: d2150c249b9643698fb987d85678a3e4 creating new one. I1102 12:55:00.621408 362 tablet_bootstrap.cc:234] Tablet 1e197352e50a4c37ad9ac16503d7dd58: No bootstrap required, opened a new log I1102 12:55:00.621769 362 ts_tablet_manager.cc:330] Time spent tablet 1e197352e50a4c37ad9ac16503d7dd58 bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:55:00.622125 365 tablet_bootstrap.cc:234] Tablet d2150c249b9643698fb987d85678a3e4: No bootstrap required, opened a new log I1102 12:55:00.622486 365 ts_tablet_manager.cc:330] Time spent tablet d2150c249b9643698fb987d85678a3e4 bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:55:00.623329 362 ts_tablet_manager.cc:350] Time spent starting tablet 1e197352e50a4c37ad9ac16503d7dd58: real 0.002s user 0.001s sys 0.000s I1102 12:55:00.624119 365 ts_tablet_manager.cc:350] Time spent starting tablet d2150c249b9643698fb987d85678a3e4: real 0.002s user 0.001s sys 0.000s I1102 12:55:00.624173 367 tablet_peer.cc:213] Configuration changed for tablet: 1e197352e50a4c37ad9ac16503d7dd58 in peer: f47b8dfad3cc41de8cf483a28bbc4cf7 Changed from: seqno: -1 local: true peers { permanent_uuid: "f47b8dfad3cc41de8cf483a28bbc4cf7" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 41900 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "f47b8dfad3cc41de8cf483a28bbc4cf7" role: LEADER last_known_addr { host: "127.0.0.1" port: 41900 } } I1102 12:55:00.624547 369 tablet_peer.cc:213] Configuration changed for tablet: d2150c249b9643698fb987d85678a3e4 in peer: f47b8dfad3cc41de8cf483a28bbc4cf7 Changed from: seqno: -1 local: true peers { permanent_uuid: "f47b8dfad3cc41de8cf483a28bbc4cf7" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 41900 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "f47b8dfad3cc41de8cf483a28bbc4cf7" role: LEADER last_known_addr { host: "127.0.0.1" port: 41900 } } I1102 12:55:00.630553 32750 catalog_manager.cc:1127] Tablet: d2150c249b9643698fb987d85678a3e4 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "f47b8dfad3cc41de8cf483a28bbc4cf7" role: LEADER last_known_addr { host: "127.0.0.1" port: 41900 } } I1102 12:55:00.633513 32750 catalog_manager.cc:1127] Tablet: 1e197352e50a4c37ad9ac16503d7dd58 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "f47b8dfad3cc41de8cf483a28bbc4cf7" role: LEADER last_known_addr { host: "127.0.0.1" port: 41900 } } I1102 12:55:00.644641 32755 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:35271: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:00.651916 332 tablet_service.cc:264] Processing CreateTablet for tablet c9fbf3df8837466b90b36bd3c7e733dd (table=client-testtb2 [id=3e53672a8f014fa3a13a78e78db2f635]), range=["", ""] I1102 12:55:00.653673 332 ts_tablet_manager.cc:460] Registered tablet c9fbf3df8837466b90b36bd3c7e733dd I1102 12:55:00.653825 362 ts_tablet_manager.cc:325] Bootstrapping tablet: c9fbf3df8837466b90b36bd3c7e733dd I1102 12:55:00.654716 362 tablet_bootstrap.cc:234] Tablet c9fbf3df8837466b90b36bd3c7e733dd: Bootstrap starting. I1102 12:55:00.658449 362 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: c9fbf3df8837466b90b36bd3c7e733dd creating new one. I1102 12:55:00.659582 362 tablet_bootstrap.cc:234] Tablet c9fbf3df8837466b90b36bd3c7e733dd: No bootstrap required, opened a new log I1102 12:55:00.659833 362 ts_tablet_manager.cc:330] Time spent tablet c9fbf3df8837466b90b36bd3c7e733dd bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:00.661047 362 ts_tablet_manager.cc:350] Time spent starting tablet c9fbf3df8837466b90b36bd3c7e733dd: real 0.001s user 0.001s sys 0.000s I1102 12:55:00.661378 374 tablet_peer.cc:213] Configuration changed for tablet: c9fbf3df8837466b90b36bd3c7e733dd in peer: f47b8dfad3cc41de8cf483a28bbc4cf7 Changed from: seqno: -1 local: true peers { permanent_uuid: "f47b8dfad3cc41de8cf483a28bbc4cf7" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 41900 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "f47b8dfad3cc41de8cf483a28bbc4cf7" role: LEADER last_known_addr { host: "127.0.0.1" port: 41900 } } I1102 12:55:00.668742 32754 catalog_manager.cc:1127] Tablet: c9fbf3df8837466b90b36bd3c7e733dd reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "f47b8dfad3cc41de8cf483a28bbc4cf7" role: LEADER last_known_addr { host: "127.0.0.1" port: 41900 } } I1102 12:55:00.691051 31819 client-test.cc:749] Successfully saw scanner close on iteration 0 I1102 12:55:00.695792 31819 client-test.cc:749] Successfully saw scanner close on iteration 1 I1102 12:55:00.700366 31819 client-test.cc:749] Successfully saw scanner close on iteration 1 I1102 12:55:00.700405 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:00.714011 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:00.714088 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:00.714135 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: d2150c249b9643698fb987d85678a3e4 I1102 12:55:00.714166 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(d2150c249b9643698fb987d85678a3e4) I1102 12:55:00.714189 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(d2150c249b9643698fb987d85678a3e4) I1102 12:55:00.714210 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(d2150c249b9643698fb987d85678a3e4) I1102 12:55:00.714875 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: c9fbf3df8837466b90b36bd3c7e733dd I1102 12:55:00.714918 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(c9fbf3df8837466b90b36bd3c7e733dd) I1102 12:55:00.714937 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(c9fbf3df8837466b90b36bd3c7e733dd) I1102 12:55:00.714959 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(c9fbf3df8837466b90b36bd3c7e733dd) I1102 12:55:00.715466 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 1e197352e50a4c37ad9ac16503d7dd58 I1102 12:55:00.715500 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(1e197352e50a4c37ad9ac16503d7dd58) I1102 12:55:00.715526 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(1e197352e50a4c37ad9ac16503d7dd58) I1102 12:55:00.715540 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(1e197352e50a4c37ad9ac16503d7dd58) I1102 12:55:00.725940 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:00.725973 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:00.726016 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:00.727411 31819 master.cc:124] Master@127.0.0.1:38427 shutting down... I1102 12:55:00.739150 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:00.739687 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:00.740104 31819 master.cc:127] Master@127.0.0.1:38427 shutdown complete. [ OK ] ClientTest.TestCloseScanner (330 ms) [ RUN ] ClientTest.TestScanTimeout I1102 12:55:00.900328 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanTimeout.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "1f7c25fd391b44a78efe19b9c9b30792" format_stamp: "Formatted at 2014-11-02 20:55:00 on qiwi" I1102 12:55:00.900560 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanTimeout.1414961696528764-31819/minicluster-data/master-0-root uuid: "1f7c25fd391b44a78efe19b9c9b30792" format_stamp: "Formatted at 2014-11-02 20:55:00 on qiwi" I1102 12:55:00.901970 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 493278 I1102 12:55:00.907456 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:00.907486 31819 webserver.cc:117] Document root disabled I1102 12:55:00.907866 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:33485 I1102 12:55:00.909986 404 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:00.913887 404 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:00.915134 404 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:00.917193 406 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 1f7c25fd391b44a78efe19b9c9b30792 Changed from: seqno: 0 local: true peers { permanent_uuid: "1f7c25fd391b44a78efe19b9c9b30792" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "1f7c25fd391b44a78efe19b9c9b30792" role: LEADER } I1102 12:55:00.917271 406 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "1f7c25fd391b44a78efe19b9c9b30792" role: LEADER } I1102 12:55:00.917841 404 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:00.920572 404 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:00.921553 404 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:00.922648 404 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:00.922933 412 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 1f7c25fd391b44a78efe19b9c9b30792 Changed from: seqno: 0 local: true peers { permanent_uuid: "1f7c25fd391b44a78efe19b9c9b30792" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "1f7c25fd391b44a78efe19b9c9b30792" role: LEADER } I1102 12:55:00.922981 404 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:00.922987 412 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "1f7c25fd391b44a78efe19b9c9b30792" role: LEADER } I1102 12:55:00.934514 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestScanTimeout.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "a690b3bea5c748b9b96eb737aa19666f" format_stamp: "Formatted at 2014-11-02 20:55:00 on qiwi" I1102 12:55:00.934628 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestScanTimeout.1414961696528764-31819/minicluster-data/ts-0-root uuid: "a690b3bea5c748b9b96eb737aa19666f" format_stamp: "Formatted at 2014-11-02 20:55:00 on qiwi" I1102 12:55:00.935439 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 493278 I1102 12:55:00.939254 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:00.939273 31819 webserver.cc:117] Document root disabled I1102 12:55:00.939508 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:44115 I1102 12:55:00.939633 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:00.941053 467 heartbeater.cc:227] Registering TS with master... I1102 12:55:00.941122 467 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:00.941429 383 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "a690b3bea5c748b9b96eb737aa19666f" instance_seqno: 1414961700938982 } with Master I1102 12:55:00.942193 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00214885s I1102 12:55:00.944946 385 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:46130: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:00.954202 439 tablet_service.cc:264] Processing CreateTablet for tablet 0596e49030df4110b4df8742ff566604 (table=client-testtb [id=abe3a147849a41d89328a945f8073cfd]), range=["\x00\x00\x00\t", ""] I1102 12:55:00.954282 440 tablet_service.cc:264] Processing CreateTablet for tablet e0940ba6ac7347b085ff3160dd07b375 (table=client-testtb [id=abe3a147849a41d89328a945f8073cfd]), range=["", "\x00\x00\x00\t"] I1102 12:55:00.955543 440 ts_tablet_manager.cc:460] Registered tablet e0940ba6ac7347b085ff3160dd07b375 I1102 12:55:00.955575 439 ts_tablet_manager.cc:460] Registered tablet 0596e49030df4110b4df8742ff566604 I1102 12:55:00.955839 476 ts_tablet_manager.cc:325] Bootstrapping tablet: e0940ba6ac7347b085ff3160dd07b375 I1102 12:55:00.956167 477 ts_tablet_manager.cc:325] Bootstrapping tablet: 0596e49030df4110b4df8742ff566604 I1102 12:55:00.956465 476 tablet_bootstrap.cc:234] Tablet e0940ba6ac7347b085ff3160dd07b375: Bootstrap starting. I1102 12:55:00.956763 477 tablet_bootstrap.cc:234] Tablet 0596e49030df4110b4df8742ff566604: Bootstrap starting. I1102 12:55:00.959300 476 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: e0940ba6ac7347b085ff3160dd07b375 creating new one. I1102 12:55:00.959692 477 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 0596e49030df4110b4df8742ff566604 creating new one. I1102 12:55:00.960479 476 tablet_bootstrap.cc:234] Tablet e0940ba6ac7347b085ff3160dd07b375: No bootstrap required, opened a new log I1102 12:55:00.960712 477 tablet_bootstrap.cc:234] Tablet 0596e49030df4110b4df8742ff566604: No bootstrap required, opened a new log I1102 12:55:00.960758 476 ts_tablet_manager.cc:330] Time spent tablet e0940ba6ac7347b085ff3160dd07b375 bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:00.960995 477 ts_tablet_manager.cc:330] Time spent tablet 0596e49030df4110b4df8742ff566604 bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:00.962282 477 ts_tablet_manager.cc:350] Time spent starting tablet 0596e49030df4110b4df8742ff566604: real 0.001s user 0.001s sys 0.000s I1102 12:55:00.962442 476 ts_tablet_manager.cc:350] Time spent starting tablet e0940ba6ac7347b085ff3160dd07b375: real 0.002s user 0.001s sys 0.000s I1102 12:55:00.963101 482 tablet_peer.cc:213] Configuration changed for tablet: e0940ba6ac7347b085ff3160dd07b375 in peer: a690b3bea5c748b9b96eb737aa19666f Changed from: seqno: -1 local: true peers { permanent_uuid: "a690b3bea5c748b9b96eb737aa19666f" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 38753 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "a690b3bea5c748b9b96eb737aa19666f" role: LEADER last_known_addr { host: "127.0.0.1" port: 38753 } } I1102 12:55:00.963122 483 tablet_peer.cc:213] Configuration changed for tablet: 0596e49030df4110b4df8742ff566604 in peer: a690b3bea5c748b9b96eb737aa19666f Changed from: seqno: -1 local: true peers { permanent_uuid: "a690b3bea5c748b9b96eb737aa19666f" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 38753 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "a690b3bea5c748b9b96eb737aa19666f" role: LEADER last_known_addr { host: "127.0.0.1" port: 38753 } } I1102 12:55:00.963703 382 catalog_manager.cc:1127] Tablet: 0596e49030df4110b4df8742ff566604 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "a690b3bea5c748b9b96eb737aa19666f" role: LEADER last_known_addr { host: "127.0.0.1" port: 38753 } } I1102 12:55:00.965829 382 catalog_manager.cc:1127] Tablet: e0940ba6ac7347b085ff3160dd07b375 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "a690b3bea5c748b9b96eb737aa19666f" role: LEADER last_known_addr { host: "127.0.0.1" port: 38753 } } I1102 12:55:00.973247 387 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:46130: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:00.979413 441 tablet_service.cc:264] Processing CreateTablet for tablet 81e4392cc4b04d3fbeb44410e23a6995 (table=client-testtb2 [id=26897c8db4324950a471dbc0cd752bc1]), range=["", ""] I1102 12:55:00.980705 441 ts_tablet_manager.cc:460] Registered tablet 81e4392cc4b04d3fbeb44410e23a6995 I1102 12:55:00.980806 477 ts_tablet_manager.cc:325] Bootstrapping tablet: 81e4392cc4b04d3fbeb44410e23a6995 I1102 12:55:00.981560 477 tablet_bootstrap.cc:234] Tablet 81e4392cc4b04d3fbeb44410e23a6995: Bootstrap starting. I1102 12:55:00.984787 477 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 81e4392cc4b04d3fbeb44410e23a6995 creating new one. I1102 12:55:00.986132 477 tablet_bootstrap.cc:234] Tablet 81e4392cc4b04d3fbeb44410e23a6995: No bootstrap required, opened a new log I1102 12:55:00.986440 477 ts_tablet_manager.cc:330] Time spent tablet 81e4392cc4b04d3fbeb44410e23a6995 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:00.987797 477 ts_tablet_manager.cc:350] Time spent starting tablet 81e4392cc4b04d3fbeb44410e23a6995: real 0.001s user 0.001s sys 0.000s I1102 12:55:00.988206 489 tablet_peer.cc:213] Configuration changed for tablet: 81e4392cc4b04d3fbeb44410e23a6995 in peer: a690b3bea5c748b9b96eb737aa19666f Changed from: seqno: -1 local: true peers { permanent_uuid: "a690b3bea5c748b9b96eb737aa19666f" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 38753 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "a690b3bea5c748b9b96eb737aa19666f" role: LEADER last_known_addr { host: "127.0.0.1" port: 38753 } } I1102 12:55:00.994108 385 catalog_manager.cc:1127] Tablet: 81e4392cc4b04d3fbeb44410e23a6995 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "a690b3bea5c748b9b96eb737aa19666f" role: LEADER last_known_addr { host: "127.0.0.1" port: 38753 } } W1102 12:55:00.999346 471 meta_cache.cc:394] GetTableLocations(client-testtb, ) failed: Timed out: Call timed out I1102 12:55:00.999501 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:01.014215 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:01.014328 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:01.014503 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 0596e49030df4110b4df8742ff566604 I1102 12:55:01.014565 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(0596e49030df4110b4df8742ff566604) I1102 12:55:01.014614 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(0596e49030df4110b4df8742ff566604) I1102 12:55:01.014650 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(0596e49030df4110b4df8742ff566604) I1102 12:55:01.015478 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 81e4392cc4b04d3fbeb44410e23a6995 I1102 12:55:01.015547 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(81e4392cc4b04d3fbeb44410e23a6995) I1102 12:55:01.015584 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(81e4392cc4b04d3fbeb44410e23a6995) I1102 12:55:01.015627 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(81e4392cc4b04d3fbeb44410e23a6995) I1102 12:55:01.016376 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: e0940ba6ac7347b085ff3160dd07b375 I1102 12:55:01.016438 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(e0940ba6ac7347b085ff3160dd07b375) I1102 12:55:01.016465 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(e0940ba6ac7347b085ff3160dd07b375) I1102 12:55:01.016494 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(e0940ba6ac7347b085ff3160dd07b375) I1102 12:55:01.023435 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:01.023473 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:01.023538 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:01.024929 31819 master.cc:124] Master@127.0.0.1:60882 shutting down... I1102 12:55:01.037142 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:01.037928 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:01.038584 31819 master.cc:127] Master@127.0.0.1:60882 shutdown complete. [ OK ] ClientTest.TestScanTimeout (303 ms) [ RUN ] ClientTest.TestInsertSingleRowManualBatch I1102 12:55:01.184049 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestInsertSingleRowManualBatch.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "062aa6c1d32e410d80c9bb6f0dddb828" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:01.184187 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestInsertSingleRowManualBatch.1414961696528764-31819/minicluster-data/master-0-root uuid: "062aa6c1d32e410d80c9bb6f0dddb828" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:01.184893 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 493778 I1102 12:55:01.188130 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:01.188148 31819 webserver.cc:117] Document root disabled I1102 12:55:01.188385 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:47034 I1102 12:55:01.189651 520 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:01.191550 520 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:01.192268 520 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:01.193588 522 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 062aa6c1d32e410d80c9bb6f0dddb828 Changed from: seqno: 0 local: true peers { permanent_uuid: "062aa6c1d32e410d80c9bb6f0dddb828" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "062aa6c1d32e410d80c9bb6f0dddb828" role: LEADER } I1102 12:55:01.193642 522 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "062aa6c1d32e410d80c9bb6f0dddb828" role: LEADER } I1102 12:55:01.194033 520 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:01.195863 520 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:01.196532 520 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:01.197518 520 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:01.197978 578 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 062aa6c1d32e410d80c9bb6f0dddb828 Changed from: seqno: 0 local: true peers { permanent_uuid: "062aa6c1d32e410d80c9bb6f0dddb828" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "062aa6c1d32e410d80c9bb6f0dddb828" role: LEADER } I1102 12:55:01.198057 520 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:01.198074 578 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "062aa6c1d32e410d80c9bb6f0dddb828" role: LEADER } I1102 12:55:01.208874 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestInsertSingleRowManualBatch.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "bd867eefd0fb4569ba59456ab635269b" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:01.208973 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestInsertSingleRowManualBatch.1414961696528764-31819/minicluster-data/ts-0-root uuid: "bd867eefd0fb4569ba59456ab635269b" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:01.209825 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 493778 I1102 12:55:01.214426 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:01.214445 31819 webserver.cc:117] Document root disabled I1102 12:55:01.214687 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:42275 I1102 12:55:01.214812 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:01.216192 622 heartbeater.cc:227] Registering TS with master... I1102 12:55:01.216243 622 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:01.216557 499 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "bd867eefd0fb4569ba59456ab635269b" instance_seqno: 1414961701214132 } with Master I1102 12:55:01.217092 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00216245s I1102 12:55:01.219969 501 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:33218: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:01.225139 599 tablet_service.cc:264] Processing CreateTablet for tablet 2573470a7bb74a17b307d5c3b1bd2ef4 (table=client-testtb [id=54a7a1dbe2cb48db9f5a6ae6e3b81f39]), range=["", "\x00\x00\x00\t"] I1102 12:55:01.225183 600 tablet_service.cc:264] Processing CreateTablet for tablet 4b90b559889c47099e29aab46b6a3402 (table=client-testtb [id=54a7a1dbe2cb48db9f5a6ae6e3b81f39]), range=["\x00\x00\x00\t", ""] I1102 12:55:01.226023 599 ts_tablet_manager.cc:460] Registered tablet 2573470a7bb74a17b307d5c3b1bd2ef4 I1102 12:55:01.226101 600 ts_tablet_manager.cc:460] Registered tablet 4b90b559889c47099e29aab46b6a3402 I1102 12:55:01.226466 631 ts_tablet_manager.cc:325] Bootstrapping tablet: 2573470a7bb74a17b307d5c3b1bd2ef4 I1102 12:55:01.226491 632 ts_tablet_manager.cc:325] Bootstrapping tablet: 4b90b559889c47099e29aab46b6a3402 I1102 12:55:01.226871 632 tablet_bootstrap.cc:234] Tablet 4b90b559889c47099e29aab46b6a3402: Bootstrap starting. I1102 12:55:01.226897 631 tablet_bootstrap.cc:234] Tablet 2573470a7bb74a17b307d5c3b1bd2ef4: Bootstrap starting. I1102 12:55:01.228754 632 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 4b90b559889c47099e29aab46b6a3402 creating new one. I1102 12:55:01.228788 631 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 2573470a7bb74a17b307d5c3b1bd2ef4 creating new one. I1102 12:55:01.230164 631 tablet_bootstrap.cc:234] Tablet 2573470a7bb74a17b307d5c3b1bd2ef4: No bootstrap required, opened a new log I1102 12:55:01.230223 632 tablet_bootstrap.cc:234] Tablet 4b90b559889c47099e29aab46b6a3402: No bootstrap required, opened a new log I1102 12:55:01.230546 631 ts_tablet_manager.cc:330] Time spent tablet 2573470a7bb74a17b307d5c3b1bd2ef4 bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:55:01.230597 632 ts_tablet_manager.cc:330] Time spent tablet 4b90b559889c47099e29aab46b6a3402 bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:55:01.231894 632 ts_tablet_manager.cc:350] Time spent starting tablet 4b90b559889c47099e29aab46b6a3402: real 0.001s user 0.001s sys 0.000s I1102 12:55:01.231914 631 ts_tablet_manager.cc:350] Time spent starting tablet 2573470a7bb74a17b307d5c3b1bd2ef4: real 0.001s user 0.001s sys 0.000s I1102 12:55:01.232373 636 tablet_peer.cc:213] Configuration changed for tablet: 4b90b559889c47099e29aab46b6a3402 in peer: bd867eefd0fb4569ba59456ab635269b Changed from: seqno: -1 local: true peers { permanent_uuid: "bd867eefd0fb4569ba59456ab635269b" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 34202 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "bd867eefd0fb4569ba59456ab635269b" role: LEADER last_known_addr { host: "127.0.0.1" port: 34202 } } I1102 12:55:01.232388 635 tablet_peer.cc:213] Configuration changed for tablet: 2573470a7bb74a17b307d5c3b1bd2ef4 in peer: bd867eefd0fb4569ba59456ab635269b Changed from: seqno: -1 local: true peers { permanent_uuid: "bd867eefd0fb4569ba59456ab635269b" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 34202 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "bd867eefd0fb4569ba59456ab635269b" role: LEADER last_known_addr { host: "127.0.0.1" port: 34202 } } I1102 12:55:01.237953 499 catalog_manager.cc:1127] Tablet: 2573470a7bb74a17b307d5c3b1bd2ef4 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "bd867eefd0fb4569ba59456ab635269b" role: LEADER last_known_addr { host: "127.0.0.1" port: 34202 } } I1102 12:55:01.239397 499 catalog_manager.cc:1127] Tablet: 4b90b559889c47099e29aab46b6a3402 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "bd867eefd0fb4569ba59456ab635269b" role: LEADER last_known_addr { host: "127.0.0.1" port: 34202 } } I1102 12:55:01.242799 503 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:33218: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:01.249238 601 tablet_service.cc:264] Processing CreateTablet for tablet f6a199ec2dc349edba8de1a086d13be8 (table=client-testtb2 [id=94b178cd0bc349d1b6f45e7257fa9907]), range=["", ""] I1102 12:55:01.252075 601 ts_tablet_manager.cc:460] Registered tablet f6a199ec2dc349edba8de1a086d13be8 I1102 12:55:01.252691 632 ts_tablet_manager.cc:325] Bootstrapping tablet: f6a199ec2dc349edba8de1a086d13be8 I1102 12:55:01.253844 632 tablet_bootstrap.cc:234] Tablet f6a199ec2dc349edba8de1a086d13be8: Bootstrap starting. I1102 12:55:01.258579 632 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: f6a199ec2dc349edba8de1a086d13be8 creating new one. I1102 12:55:01.259804 632 tablet_bootstrap.cc:234] Tablet f6a199ec2dc349edba8de1a086d13be8: No bootstrap required, opened a new log I1102 12:55:01.260104 632 ts_tablet_manager.cc:330] Time spent tablet f6a199ec2dc349edba8de1a086d13be8 bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:55:01.261283 632 ts_tablet_manager.cc:350] Time spent starting tablet f6a199ec2dc349edba8de1a086d13be8: real 0.001s user 0.001s sys 0.000s I1102 12:55:01.261643 642 tablet_peer.cc:213] Configuration changed for tablet: f6a199ec2dc349edba8de1a086d13be8 in peer: bd867eefd0fb4569ba59456ab635269b Changed from: seqno: -1 local: true peers { permanent_uuid: "bd867eefd0fb4569ba59456ab635269b" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 34202 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "bd867eefd0fb4569ba59456ab635269b" role: LEADER last_known_addr { host: "127.0.0.1" port: 34202 } } I1102 12:55:01.263324 501 catalog_manager.cc:1127] Tablet: f6a199ec2dc349edba8de1a086d13be8 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "bd867eefd0fb4569ba59456ab635269b" role: LEADER last_known_addr { host: "127.0.0.1" port: 34202 } } W1102 12:55:01.270701 626 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:34202, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} I1102 12:55:01.273994 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:01.287091 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:01.287178 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:01.287333 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 2573470a7bb74a17b307d5c3b1bd2ef4 I1102 12:55:01.287367 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(2573470a7bb74a17b307d5c3b1bd2ef4) I1102 12:55:01.287392 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(2573470a7bb74a17b307d5c3b1bd2ef4) I1102 12:55:01.287407 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(2573470a7bb74a17b307d5c3b1bd2ef4) I1102 12:55:01.287988 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: f6a199ec2dc349edba8de1a086d13be8 I1102 12:55:01.288015 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(f6a199ec2dc349edba8de1a086d13be8) I1102 12:55:01.288038 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(f6a199ec2dc349edba8de1a086d13be8) I1102 12:55:01.288053 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(f6a199ec2dc349edba8de1a086d13be8) I1102 12:55:01.288516 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 4b90b559889c47099e29aab46b6a3402 I1102 12:55:01.288543 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(4b90b559889c47099e29aab46b6a3402) I1102 12:55:01.288564 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(4b90b559889c47099e29aab46b6a3402) I1102 12:55:01.288578 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(4b90b559889c47099e29aab46b6a3402) I1102 12:55:01.296562 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:01.296597 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:01.296650 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:01.297864 31819 master.cc:124] Master@127.0.0.1:54853 shutting down... I1102 12:55:01.310695 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:01.311709 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:01.312620 31819 master.cc:127] Master@127.0.0.1:54853 shutdown complete. [ OK ] ClientTest.TestInsertSingleRowManualBatch (270 ms) [ RUN ] ClientTest.TestAsyncFlushResponseAfterSessionDropped I1102 12:55:01.471582 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestAsyncFlushResponseAfterSessionDropped.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "11863ad48f3c40fabe54e0889e71cf40" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:01.471746 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestAsyncFlushResponseAfterSessionDropped.1414961696528764-31819/minicluster-data/master-0-root uuid: "11863ad48f3c40fabe54e0889e71cf40" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:01.472674 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 493778 I1102 12:55:01.477468 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:01.477485 31819 webserver.cc:117] Document root disabled I1102 12:55:01.477710 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:39163 I1102 12:55:01.478888 771 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:01.480810 771 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:01.481701 771 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:01.483078 773 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 11863ad48f3c40fabe54e0889e71cf40 Changed from: seqno: 0 local: true peers { permanent_uuid: "11863ad48f3c40fabe54e0889e71cf40" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "11863ad48f3c40fabe54e0889e71cf40" role: LEADER } I1102 12:55:01.483134 773 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "11863ad48f3c40fabe54e0889e71cf40" role: LEADER } I1102 12:55:01.483557 771 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:01.485695 771 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:01.486599 771 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:01.487774 771 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:01.488067 777 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 11863ad48f3c40fabe54e0889e71cf40 Changed from: seqno: 0 local: true peers { permanent_uuid: "11863ad48f3c40fabe54e0889e71cf40" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "11863ad48f3c40fabe54e0889e71cf40" role: LEADER } I1102 12:55:01.488121 771 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:01.488128 777 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "11863ad48f3c40fabe54e0889e71cf40" role: LEADER } I1102 12:55:01.499078 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestAsyncFlushResponseAfterSessionDropped.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "a8c8b386624e4320a4ea969247a6a14c" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:01.499188 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestAsyncFlushResponseAfterSessionDropped.1414961696528764-31819/minicluster-data/ts-0-root uuid: "a8c8b386624e4320a4ea969247a6a14c" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:01.500131 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 493778 I1102 12:55:01.504900 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:01.504921 31819 webserver.cc:117] Document root disabled I1102 12:55:01.505190 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:43234 I1102 12:55:01.505321 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:01.506739 820 heartbeater.cc:227] Registering TS with master... I1102 12:55:01.506798 820 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:01.507140 662 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "a8c8b386624e4320a4ea969247a6a14c" instance_seqno: 1414961701504616 } with Master I1102 12:55:01.507684 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00219121s I1102 12:55:01.510570 664 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:58203: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:01.515741 798 tablet_service.cc:264] Processing CreateTablet for tablet 449fc0f2541e4a3781e4abd496a1874c (table=client-testtb [id=346817a80628402cba5e9563f11e1bcb]), range=["", "\x00\x00\x00\t"] I1102 12:55:01.515802 799 tablet_service.cc:264] Processing CreateTablet for tablet b63ba6681236470e932bdea2f31ca983 (table=client-testtb [id=346817a80628402cba5e9563f11e1bcb]), range=["\x00\x00\x00\t", ""] I1102 12:55:01.516649 798 ts_tablet_manager.cc:460] Registered tablet 449fc0f2541e4a3781e4abd496a1874c I1102 12:55:01.516746 799 ts_tablet_manager.cc:460] Registered tablet b63ba6681236470e932bdea2f31ca983 I1102 12:55:01.517034 829 ts_tablet_manager.cc:325] Bootstrapping tablet: 449fc0f2541e4a3781e4abd496a1874c I1102 12:55:01.517066 830 ts_tablet_manager.cc:325] Bootstrapping tablet: b63ba6681236470e932bdea2f31ca983 I1102 12:55:01.517415 829 tablet_bootstrap.cc:234] Tablet 449fc0f2541e4a3781e4abd496a1874c: Bootstrap starting. I1102 12:55:01.517434 830 tablet_bootstrap.cc:234] Tablet b63ba6681236470e932bdea2f31ca983: Bootstrap starting. I1102 12:55:01.519531 829 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 449fc0f2541e4a3781e4abd496a1874c creating new one. I1102 12:55:01.519572 830 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: b63ba6681236470e932bdea2f31ca983 creating new one. I1102 12:55:01.520784 829 tablet_bootstrap.cc:234] Tablet 449fc0f2541e4a3781e4abd496a1874c: No bootstrap required, opened a new log I1102 12:55:01.520812 830 tablet_bootstrap.cc:234] Tablet b63ba6681236470e932bdea2f31ca983: No bootstrap required, opened a new log I1102 12:55:01.520975 829 ts_tablet_manager.cc:330] Time spent tablet 449fc0f2541e4a3781e4abd496a1874c bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:55:01.520997 830 ts_tablet_manager.cc:330] Time spent tablet b63ba6681236470e932bdea2f31ca983 bootstrap: real 0.004s user 0.002s sys 0.001s I1102 12:55:01.522351 830 ts_tablet_manager.cc:350] Time spent starting tablet b63ba6681236470e932bdea2f31ca983: real 0.001s user 0.001s sys 0.000s I1102 12:55:01.522373 829 ts_tablet_manager.cc:350] Time spent starting tablet 449fc0f2541e4a3781e4abd496a1874c: real 0.001s user 0.001s sys 0.000s I1102 12:55:01.522941 833 tablet_peer.cc:213] Configuration changed for tablet: 449fc0f2541e4a3781e4abd496a1874c in peer: a8c8b386624e4320a4ea969247a6a14c Changed from: seqno: -1 local: true peers { permanent_uuid: "a8c8b386624e4320a4ea969247a6a14c" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 52229 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "a8c8b386624e4320a4ea969247a6a14c" role: LEADER last_known_addr { host: "127.0.0.1" port: 52229 } } I1102 12:55:01.522963 834 tablet_peer.cc:213] Configuration changed for tablet: b63ba6681236470e932bdea2f31ca983 in peer: a8c8b386624e4320a4ea969247a6a14c Changed from: seqno: -1 local: true peers { permanent_uuid: "a8c8b386624e4320a4ea969247a6a14c" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 52229 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "a8c8b386624e4320a4ea969247a6a14c" role: LEADER last_known_addr { host: "127.0.0.1" port: 52229 } } I1102 12:55:01.528568 662 catalog_manager.cc:1127] Tablet: 449fc0f2541e4a3781e4abd496a1874c reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "a8c8b386624e4320a4ea969247a6a14c" role: LEADER last_known_addr { host: "127.0.0.1" port: 52229 } } I1102 12:55:01.530298 662 catalog_manager.cc:1127] Tablet: b63ba6681236470e932bdea2f31ca983 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "a8c8b386624e4320a4ea969247a6a14c" role: LEADER last_known_addr { host: "127.0.0.1" port: 52229 } } I1102 12:55:01.542223 667 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:58203: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:01.550052 800 tablet_service.cc:264] Processing CreateTablet for tablet 5ef5019faf884d08bcb7d621e054cad5 (table=client-testtb2 [id=cea2654a1f474073a22efec487d3b48f]), range=["", ""] I1102 12:55:01.551420 800 ts_tablet_manager.cc:460] Registered tablet 5ef5019faf884d08bcb7d621e054cad5 I1102 12:55:01.551574 830 ts_tablet_manager.cc:325] Bootstrapping tablet: 5ef5019faf884d08bcb7d621e054cad5 I1102 12:55:01.552217 830 tablet_bootstrap.cc:234] Tablet 5ef5019faf884d08bcb7d621e054cad5: Bootstrap starting. I1102 12:55:01.556015 830 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 5ef5019faf884d08bcb7d621e054cad5 creating new one. I1102 12:55:01.557432 830 tablet_bootstrap.cc:234] Tablet 5ef5019faf884d08bcb7d621e054cad5: No bootstrap required, opened a new log I1102 12:55:01.557641 830 ts_tablet_manager.cc:330] Time spent tablet 5ef5019faf884d08bcb7d621e054cad5 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:01.558660 830 ts_tablet_manager.cc:350] Time spent starting tablet 5ef5019faf884d08bcb7d621e054cad5: real 0.001s user 0.001s sys 0.000s I1102 12:55:01.558924 840 tablet_peer.cc:213] Configuration changed for tablet: 5ef5019faf884d08bcb7d621e054cad5 in peer: a8c8b386624e4320a4ea969247a6a14c Changed from: seqno: -1 local: true peers { permanent_uuid: "a8c8b386624e4320a4ea969247a6a14c" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 52229 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "a8c8b386624e4320a4ea969247a6a14c" role: LEADER last_known_addr { host: "127.0.0.1" port: 52229 } } I1102 12:55:01.565076 666 catalog_manager.cc:1127] Tablet: 5ef5019faf884d08bcb7d621e054cad5 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "a8c8b386624e4320a4ea969247a6a14c" role: LEADER last_known_addr { host: "127.0.0.1" port: 52229 } } W1102 12:55:01.569517 827 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:52229, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:01.572209 827 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:52229, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} I1102 12:55:01.574813 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:01.594123 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:01.594372 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:01.594514 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 449fc0f2541e4a3781e4abd496a1874c I1102 12:55:01.594671 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(449fc0f2541e4a3781e4abd496a1874c) I1102 12:55:01.594725 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(449fc0f2541e4a3781e4abd496a1874c) I1102 12:55:01.594777 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(449fc0f2541e4a3781e4abd496a1874c) I1102 12:55:01.596659 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 5ef5019faf884d08bcb7d621e054cad5 I1102 12:55:01.596751 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(5ef5019faf884d08bcb7d621e054cad5) I1102 12:55:01.596793 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(5ef5019faf884d08bcb7d621e054cad5) I1102 12:55:01.596837 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(5ef5019faf884d08bcb7d621e054cad5) I1102 12:55:01.597967 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: b63ba6681236470e932bdea2f31ca983 I1102 12:55:01.598054 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(b63ba6681236470e932bdea2f31ca983) I1102 12:55:01.598093 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(b63ba6681236470e932bdea2f31ca983) I1102 12:55:01.598137 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(b63ba6681236470e932bdea2f31ca983) I1102 12:55:01.606968 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:01.606997 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:01.607029 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:01.607996 31819 master.cc:124] Master@127.0.0.1:35070 shutting down... I1102 12:55:01.622262 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:01.622810 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:01.623253 31819 master.cc:127] Master@127.0.0.1:35070 shutdown complete. [ OK ] ClientTest.TestAsyncFlushResponseAfterSessionDropped (311 ms) [ RUN ] ClientTest.TestMultipleMultiRowManualBatches I1102 12:55:01.771577 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestMultipleMultiRowManualBatches.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "a9381e8d37a74e64965de4d55f7ebf3a" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:01.771708 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestMultipleMultiRowManualBatches.1414961696528764-31819/minicluster-data/master-0-root uuid: "a9381e8d37a74e64965de4d55f7ebf3a" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:01.772464 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 493778 I1102 12:55:01.775445 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:01.775465 31819 webserver.cc:117] Document root disabled I1102 12:55:01.775696 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:58555 I1102 12:55:01.777001 872 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:01.778986 872 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:01.779799 872 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:01.781152 959 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: a9381e8d37a74e64965de4d55f7ebf3a Changed from: seqno: 0 local: true peers { permanent_uuid: "a9381e8d37a74e64965de4d55f7ebf3a" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "a9381e8d37a74e64965de4d55f7ebf3a" role: LEADER } I1102 12:55:01.781254 959 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "a9381e8d37a74e64965de4d55f7ebf3a" role: LEADER } I1102 12:55:01.781589 872 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:01.783540 872 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:01.784394 872 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:01.785537 872 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:01.785922 963 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: a9381e8d37a74e64965de4d55f7ebf3a Changed from: seqno: 0 local: true peers { permanent_uuid: "a9381e8d37a74e64965de4d55f7ebf3a" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "a9381e8d37a74e64965de4d55f7ebf3a" role: LEADER } I1102 12:55:01.786001 872 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:01.786015 963 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "a9381e8d37a74e64965de4d55f7ebf3a" role: LEADER } I1102 12:55:01.802225 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestMultipleMultiRowManualBatches.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "295416af6ecc4e30947e450627ead2ad" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:01.802438 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestMultipleMultiRowManualBatches.1414961696528764-31819/minicluster-data/ts-0-root uuid: "295416af6ecc4e30947e450627ead2ad" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:01.804044 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 493778 I1102 12:55:01.812851 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:01.812904 31819 webserver.cc:117] Document root disabled I1102 12:55:01.813321 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:43213 I1102 12:55:01.813554 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:01.815697 1009 heartbeater.cc:227] Registering TS with master... I1102 12:55:01.815788 1009 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:01.816290 850 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "295416af6ecc4e30947e450627ead2ad" instance_seqno: 1414961701812311 } with Master I1102 12:55:01.817028 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00326925s I1102 12:55:01.821187 852 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:58626: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:01.829303 987 tablet_service.cc:264] Processing CreateTablet for tablet 248cea978ffc405ca83687c4a7208892 (table=client-testtb [id=dfd2e4b5fa214d578c4233f819591109]), range=["\x00\x00\x00\t", ""] I1102 12:55:01.829370 988 tablet_service.cc:264] Processing CreateTablet for tablet 3206df2b25e440df92726e09ab9716fd (table=client-testtb [id=dfd2e4b5fa214d578c4233f819591109]), range=["", "\x00\x00\x00\t"] I1102 12:55:01.830584 987 ts_tablet_manager.cc:460] Registered tablet 248cea978ffc405ca83687c4a7208892 I1102 12:55:01.830620 988 ts_tablet_manager.cc:460] Registered tablet 3206df2b25e440df92726e09ab9716fd I1102 12:55:01.831220 1018 ts_tablet_manager.cc:325] Bootstrapping tablet: 248cea978ffc405ca83687c4a7208892 I1102 12:55:01.831236 1019 ts_tablet_manager.cc:325] Bootstrapping tablet: 3206df2b25e440df92726e09ab9716fd I1102 12:55:01.831858 1019 tablet_bootstrap.cc:234] Tablet 3206df2b25e440df92726e09ab9716fd: Bootstrap starting. I1102 12:55:01.831858 1018 tablet_bootstrap.cc:234] Tablet 248cea978ffc405ca83687c4a7208892: Bootstrap starting. I1102 12:55:01.835360 1018 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 248cea978ffc405ca83687c4a7208892 creating new one. I1102 12:55:01.835371 1019 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 3206df2b25e440df92726e09ab9716fd creating new one. I1102 12:55:01.837435 1018 tablet_bootstrap.cc:234] Tablet 248cea978ffc405ca83687c4a7208892: No bootstrap required, opened a new log I1102 12:55:01.837431 1019 tablet_bootstrap.cc:234] Tablet 3206df2b25e440df92726e09ab9716fd: No bootstrap required, opened a new log I1102 12:55:01.837779 1018 ts_tablet_manager.cc:330] Time spent tablet 248cea978ffc405ca83687c4a7208892 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:01.837950 1019 ts_tablet_manager.cc:330] Time spent tablet 3206df2b25e440df92726e09ab9716fd bootstrap: real 0.007s user 0.004s sys 0.001s I1102 12:55:01.839372 1018 ts_tablet_manager.cc:350] Time spent starting tablet 248cea978ffc405ca83687c4a7208892: real 0.001s user 0.001s sys 0.000s I1102 12:55:01.839687 1019 ts_tablet_manager.cc:350] Time spent starting tablet 3206df2b25e440df92726e09ab9716fd: real 0.002s user 0.001s sys 0.000s I1102 12:55:01.840065 1023 tablet_peer.cc:213] Configuration changed for tablet: 248cea978ffc405ca83687c4a7208892 in peer: 295416af6ecc4e30947e450627ead2ad Changed from: seqno: -1 local: true peers { permanent_uuid: "295416af6ecc4e30947e450627ead2ad" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 36243 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "295416af6ecc4e30947e450627ead2ad" role: LEADER last_known_addr { host: "127.0.0.1" port: 36243 } } I1102 12:55:01.840353 1022 tablet_peer.cc:213] Configuration changed for tablet: 3206df2b25e440df92726e09ab9716fd in peer: 295416af6ecc4e30947e450627ead2ad Changed from: seqno: -1 local: true peers { permanent_uuid: "295416af6ecc4e30947e450627ead2ad" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 36243 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "295416af6ecc4e30947e450627ead2ad" role: LEADER last_known_addr { host: "127.0.0.1" port: 36243 } } I1102 12:55:01.849464 852 catalog_manager.cc:1127] Tablet: 248cea978ffc405ca83687c4a7208892 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "295416af6ecc4e30947e450627ead2ad" role: LEADER last_known_addr { host: "127.0.0.1" port: 36243 } } I1102 12:55:01.851994 852 catalog_manager.cc:1127] Tablet: 3206df2b25e440df92726e09ab9716fd reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "295416af6ecc4e30947e450627ead2ad" role: LEADER last_known_addr { host: "127.0.0.1" port: 36243 } } I1102 12:55:01.862517 858 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:58626: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:01.872220 989 tablet_service.cc:264] Processing CreateTablet for tablet 83351ddf7ee54e7994805a9a1016c53b (table=client-testtb2 [id=7e7e4e10790d418ab22278edcbfa504c]), range=["", ""] I1102 12:55:01.874253 989 ts_tablet_manager.cc:460] Registered tablet 83351ddf7ee54e7994805a9a1016c53b I1102 12:55:01.874493 1018 ts_tablet_manager.cc:325] Bootstrapping tablet: 83351ddf7ee54e7994805a9a1016c53b I1102 12:55:01.875529 1018 tablet_bootstrap.cc:234] Tablet 83351ddf7ee54e7994805a9a1016c53b: Bootstrap starting. I1102 12:55:01.880381 1018 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 83351ddf7ee54e7994805a9a1016c53b creating new one. I1102 12:55:01.882179 1018 tablet_bootstrap.cc:234] Tablet 83351ddf7ee54e7994805a9a1016c53b: No bootstrap required, opened a new log I1102 12:55:01.882647 1018 ts_tablet_manager.cc:330] Time spent tablet 83351ddf7ee54e7994805a9a1016c53b bootstrap: real 0.008s user 0.006s sys 0.001s I1102 12:55:01.884327 1018 ts_tablet_manager.cc:350] Time spent starting tablet 83351ddf7ee54e7994805a9a1016c53b: real 0.002s user 0.001s sys 0.000s I1102 12:55:01.884742 1029 tablet_peer.cc:213] Configuration changed for tablet: 83351ddf7ee54e7994805a9a1016c53b in peer: 295416af6ecc4e30947e450627ead2ad Changed from: seqno: -1 local: true peers { permanent_uuid: "295416af6ecc4e30947e450627ead2ad" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 36243 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "295416af6ecc4e30947e450627ead2ad" role: LEADER last_known_addr { host: "127.0.0.1" port: 36243 } } I1102 12:55:01.888407 856 catalog_manager.cc:1127] Tablet: 83351ddf7ee54e7994805a9a1016c53b reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "295416af6ecc4e30947e450627ead2ad" role: LEADER last_known_addr { host: "127.0.0.1" port: 36243 } } W1102 12:55:01.898522 1013 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:36243, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:01.898983 1013 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:36243, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:01.905431 1013 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:36243, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:01.905623 1013 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:36243, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:01.911020 1013 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:36243, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:01.911193 1013 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:36243, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:01.917417 1013 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:36243, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:01.917624 1013 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:36243, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:01.923333 1013 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:36243, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:01.923480 1013 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:36243, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} I1102 12:55:01.938894 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:01.955070 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:01.955224 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:01.955468 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 83351ddf7ee54e7994805a9a1016c53b I1102 12:55:01.955531 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(83351ddf7ee54e7994805a9a1016c53b) I1102 12:55:01.955580 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(83351ddf7ee54e7994805a9a1016c53b) I1102 12:55:01.955615 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(83351ddf7ee54e7994805a9a1016c53b) I1102 12:55:01.956683 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 248cea978ffc405ca83687c4a7208892 I1102 12:55:01.956740 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(248cea978ffc405ca83687c4a7208892) I1102 12:55:01.956786 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(248cea978ffc405ca83687c4a7208892) I1102 12:55:01.956814 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(248cea978ffc405ca83687c4a7208892) I1102 12:55:01.958078 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 3206df2b25e440df92726e09ab9716fd I1102 12:55:01.958145 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(3206df2b25e440df92726e09ab9716fd) I1102 12:55:01.958200 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(3206df2b25e440df92726e09ab9716fd) I1102 12:55:01.958232 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(3206df2b25e440df92726e09ab9716fd) I1102 12:55:01.967008 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:01.967030 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:01.967068 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:01.968125 31819 master.cc:124] Master@127.0.0.1:44058 shutting down... I1102 12:55:01.982486 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:01.983639 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:01.984678 31819 master.cc:127] Master@127.0.0.1:44058 shutdown complete. [ OK ] ClientTest.TestMultipleMultiRowManualBatches (363 ms) [ RUN ] ClientTest.TestBatchWithPartialError I1102 12:55:02.144799 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestBatchWithPartialError.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "4a43545b424b49d686505354d0a4f505" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:02.144909 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestBatchWithPartialError.1414961696528764-31819/minicluster-data/master-0-root uuid: "4a43545b424b49d686505354d0a4f505" format_stamp: "Formatted at 2014-11-02 20:55:01 on qiwi" I1102 12:55:02.146736 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 494278 I1102 12:55:02.152876 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:02.152901 31819 webserver.cc:117] Document root disabled I1102 12:55:02.153229 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:37005 I1102 12:55:02.154958 1058 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:02.158643 1058 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:02.159888 1058 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:02.161802 1060 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 4a43545b424b49d686505354d0a4f505 Changed from: seqno: 0 local: true peers { permanent_uuid: "4a43545b424b49d686505354d0a4f505" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "4a43545b424b49d686505354d0a4f505" role: LEADER } I1102 12:55:02.161877 1060 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "4a43545b424b49d686505354d0a4f505" role: LEADER } I1102 12:55:02.162472 1058 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:02.165796 1058 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:02.167153 1058 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:02.168545 1058 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:02.169031 1065 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 4a43545b424b49d686505354d0a4f505 Changed from: seqno: 0 local: true peers { permanent_uuid: "4a43545b424b49d686505354d0a4f505" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "4a43545b424b49d686505354d0a4f505" role: LEADER } I1102 12:55:02.169136 1058 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:02.169141 1065 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "4a43545b424b49d686505354d0a4f505" role: LEADER } I1102 12:55:02.181818 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestBatchWithPartialError.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "638970e39323436383f5cb1adbc996aa" format_stamp: "Formatted at 2014-11-02 20:55:02 on qiwi" I1102 12:55:02.182205 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestBatchWithPartialError.1414961696528764-31819/minicluster-data/ts-0-root uuid: "638970e39323436383f5cb1adbc996aa" format_stamp: "Formatted at 2014-11-02 20:55:02 on qiwi" I1102 12:55:02.185004 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 494278 I1102 12:55:02.192301 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:02.192330 31819 webserver.cc:117] Document root disabled I1102 12:55:02.192736 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:56851 I1102 12:55:02.192961 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:02.194943 1109 heartbeater.cc:227] Registering TS with master... I1102 12:55:02.195044 1109 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:02.195581 1037 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "638970e39323436383f5cb1adbc996aa" instance_seqno: 1414961702191852 } with Master I1102 12:55:02.196482 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00327186s I1102 12:55:02.200494 1039 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:59112: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:02.208500 1086 tablet_service.cc:264] Processing CreateTablet for tablet 151c90303b8f4088a55cbca12a09339c (table=client-testtb [id=ea524e9cd9a44281807351be001c36ba]), range=["", "\x00\x00\x00\t"] I1102 12:55:02.208562 1087 tablet_service.cc:264] Processing CreateTablet for tablet cba331aff2c94de380a9d91991fa07f8 (table=client-testtb [id=ea524e9cd9a44281807351be001c36ba]), range=["\x00\x00\x00\t", ""] I1102 12:55:02.209856 1086 ts_tablet_manager.cc:460] Registered tablet 151c90303b8f4088a55cbca12a09339c I1102 12:55:02.209992 1087 ts_tablet_manager.cc:460] Registered tablet cba331aff2c94de380a9d91991fa07f8 I1102 12:55:02.210180 1118 ts_tablet_manager.cc:325] Bootstrapping tablet: 151c90303b8f4088a55cbca12a09339c I1102 12:55:02.210448 1119 ts_tablet_manager.cc:325] Bootstrapping tablet: cba331aff2c94de380a9d91991fa07f8 I1102 12:55:02.210928 1118 tablet_bootstrap.cc:234] Tablet 151c90303b8f4088a55cbca12a09339c: Bootstrap starting. I1102 12:55:02.211041 1119 tablet_bootstrap.cc:234] Tablet cba331aff2c94de380a9d91991fa07f8: Bootstrap starting. I1102 12:55:02.213971 1118 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 151c90303b8f4088a55cbca12a09339c creating new one. I1102 12:55:02.214215 1119 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: cba331aff2c94de380a9d91991fa07f8 creating new one. I1102 12:55:02.215443 1118 tablet_bootstrap.cc:234] Tablet 151c90303b8f4088a55cbca12a09339c: No bootstrap required, opened a new log I1102 12:55:02.215550 1119 tablet_bootstrap.cc:234] Tablet cba331aff2c94de380a9d91991fa07f8: No bootstrap required, opened a new log I1102 12:55:02.216032 1118 ts_tablet_manager.cc:330] Time spent tablet 151c90303b8f4088a55cbca12a09339c bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:02.216130 1119 ts_tablet_manager.cc:330] Time spent tablet cba331aff2c94de380a9d91991fa07f8 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:02.217988 1119 ts_tablet_manager.cc:350] Time spent starting tablet cba331aff2c94de380a9d91991fa07f8: real 0.002s user 0.001s sys 0.000s I1102 12:55:02.218016 1118 ts_tablet_manager.cc:350] Time spent starting tablet 151c90303b8f4088a55cbca12a09339c: real 0.002s user 0.002s sys 0.001s I1102 12:55:02.218524 1122 tablet_peer.cc:213] Configuration changed for tablet: 151c90303b8f4088a55cbca12a09339c in peer: 638970e39323436383f5cb1adbc996aa Changed from: seqno: -1 local: true peers { permanent_uuid: "638970e39323436383f5cb1adbc996aa" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 34198 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "638970e39323436383f5cb1adbc996aa" role: LEADER last_known_addr { host: "127.0.0.1" port: 34198 } } I1102 12:55:02.218816 1123 tablet_peer.cc:213] Configuration changed for tablet: cba331aff2c94de380a9d91991fa07f8 in peer: 638970e39323436383f5cb1adbc996aa Changed from: seqno: -1 local: true peers { permanent_uuid: "638970e39323436383f5cb1adbc996aa" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 34198 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "638970e39323436383f5cb1adbc996aa" role: LEADER last_known_addr { host: "127.0.0.1" port: 34198 } } I1102 12:55:02.228399 1039 catalog_manager.cc:1127] Tablet: 151c90303b8f4088a55cbca12a09339c reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "638970e39323436383f5cb1adbc996aa" role: LEADER last_known_addr { host: "127.0.0.1" port: 34198 } } I1102 12:55:02.230651 1039 catalog_manager.cc:1127] Tablet: cba331aff2c94de380a9d91991fa07f8 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "638970e39323436383f5cb1adbc996aa" role: LEADER last_known_addr { host: "127.0.0.1" port: 34198 } } I1102 12:55:02.239497 1044 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:59112: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:02.246387 1088 tablet_service.cc:264] Processing CreateTablet for tablet c28618ae54b9425e9f9f07500675cdd3 (table=client-testtb2 [id=f1983545aad64d68b81c89e842e8bd61]), range=["", ""] I1102 12:55:02.247663 1088 ts_tablet_manager.cc:460] Registered tablet c28618ae54b9425e9f9f07500675cdd3 I1102 12:55:02.247792 1119 ts_tablet_manager.cc:325] Bootstrapping tablet: c28618ae54b9425e9f9f07500675cdd3 I1102 12:55:02.248384 1119 tablet_bootstrap.cc:234] Tablet c28618ae54b9425e9f9f07500675cdd3: Bootstrap starting. I1102 12:55:02.251766 1119 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: c28618ae54b9425e9f9f07500675cdd3 creating new one. I1102 12:55:02.253324 1119 tablet_bootstrap.cc:234] Tablet c28618ae54b9425e9f9f07500675cdd3: No bootstrap required, opened a new log I1102 12:55:02.253801 1119 ts_tablet_manager.cc:330] Time spent tablet c28618ae54b9425e9f9f07500675cdd3 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:02.255290 1119 ts_tablet_manager.cc:350] Time spent starting tablet c28618ae54b9425e9f9f07500675cdd3: real 0.001s user 0.001s sys 0.000s I1102 12:55:02.255724 1133 tablet_peer.cc:213] Configuration changed for tablet: c28618ae54b9425e9f9f07500675cdd3 in peer: 638970e39323436383f5cb1adbc996aa Changed from: seqno: -1 local: true peers { permanent_uuid: "638970e39323436383f5cb1adbc996aa" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 34198 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "638970e39323436383f5cb1adbc996aa" role: LEADER last_known_addr { host: "127.0.0.1" port: 34198 } } I1102 12:55:02.265450 1045 catalog_manager.cc:1127] Tablet: c28618ae54b9425e9f9f07500675cdd3 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "638970e39323436383f5cb1adbc996aa" role: LEADER last_known_addr { host: "127.0.0.1" port: 34198 } } W1102 12:55:02.277355 1113 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:34198, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:02.280766 1113 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:34198, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} I1102 12:55:02.286180 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:02.299057 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:02.299154 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:02.299223 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: c28618ae54b9425e9f9f07500675cdd3 I1102 12:55:02.299293 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(c28618ae54b9425e9f9f07500675cdd3) I1102 12:55:02.299311 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(c28618ae54b9425e9f9f07500675cdd3) I1102 12:55:02.299330 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(c28618ae54b9425e9f9f07500675cdd3) I1102 12:55:02.299845 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 151c90303b8f4088a55cbca12a09339c I1102 12:55:02.299877 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(151c90303b8f4088a55cbca12a09339c) I1102 12:55:02.299891 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(151c90303b8f4088a55cbca12a09339c) I1102 12:55:02.299909 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(151c90303b8f4088a55cbca12a09339c) I1102 12:55:02.300295 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: cba331aff2c94de380a9d91991fa07f8 I1102 12:55:02.300325 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(cba331aff2c94de380a9d91991fa07f8) I1102 12:55:02.300339 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(cba331aff2c94de380a9d91991fa07f8) I1102 12:55:02.300356 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(cba331aff2c94de380a9d91991fa07f8) I1102 12:55:02.304605 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:02.304648 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:02.304697 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:02.305869 31819 master.cc:124] Master@127.0.0.1:52640 shutting down... I1102 12:55:02.317855 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:02.318438 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:02.318882 31819 master.cc:127] Master@127.0.0.1:52640 shutdown complete. [ OK ] ClientTest.TestBatchWithPartialError (331 ms) [ RUN ] ClientTest.TestEmptyBatch I1102 12:55:02.467401 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestEmptyBatch.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "c6535a94f51841b6ab5f9f59c248c4ed" format_stamp: "Formatted at 2014-11-02 20:55:02 on qiwi" I1102 12:55:02.467530 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestEmptyBatch.1414961696528764-31819/minicluster-data/master-0-root uuid: "c6535a94f51841b6ab5f9f59c248c4ed" format_stamp: "Formatted at 2014-11-02 20:55:02 on qiwi" I1102 12:55:02.468405 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 494278 I1102 12:55:02.471288 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:02.471318 31819 webserver.cc:117] Document root disabled I1102 12:55:02.471554 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:59723 I1102 12:55:02.472795 1166 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:02.474663 1166 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:02.475483 1166 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:02.476908 1168 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: c6535a94f51841b6ab5f9f59c248c4ed Changed from: seqno: 0 local: true peers { permanent_uuid: "c6535a94f51841b6ab5f9f59c248c4ed" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "c6535a94f51841b6ab5f9f59c248c4ed" role: LEADER } I1102 12:55:02.476974 1168 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "c6535a94f51841b6ab5f9f59c248c4ed" role: LEADER } I1102 12:55:02.477673 1166 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:02.479611 1166 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:02.480434 1166 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:02.481458 1166 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:02.481746 1172 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: c6535a94f51841b6ab5f9f59c248c4ed Changed from: seqno: 0 local: true peers { permanent_uuid: "c6535a94f51841b6ab5f9f59c248c4ed" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "c6535a94f51841b6ab5f9f59c248c4ed" role: LEADER } I1102 12:55:02.481804 1166 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:02.481806 1172 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "c6535a94f51841b6ab5f9f59c248c4ed" role: LEADER } I1102 12:55:02.493085 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestEmptyBatch.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "06fbd87001d34eef866cffdcc74c8ded" format_stamp: "Formatted at 2014-11-02 20:55:02 on qiwi" I1102 12:55:02.493193 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestEmptyBatch.1414961696528764-31819/minicluster-data/ts-0-root uuid: "06fbd87001d34eef866cffdcc74c8ded" format_stamp: "Formatted at 2014-11-02 20:55:02 on qiwi" I1102 12:55:02.494061 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 494278 I1102 12:55:02.498412 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:02.498438 31819 webserver.cc:117] Document root disabled I1102 12:55:02.498667 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:40993 I1102 12:55:02.498801 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:02.500247 1224 heartbeater.cc:227] Registering TS with master... I1102 12:55:02.500411 1224 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:02.500741 1142 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "06fbd87001d34eef866cffdcc74c8ded" instance_seqno: 1414961702498147 } with Master I1102 12:55:02.501155 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00218171s I1102 12:55:02.504045 1147 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:56785: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:02.509326 1198 tablet_service.cc:264] Processing CreateTablet for tablet ce1823b844da44989af4eb1deb03b4b0 (table=client-testtb [id=c065a27ae5044893b5d8ff341586a52a]), range=["\x00\x00\x00\t", ""] I1102 12:55:02.509346 1199 tablet_service.cc:264] Processing CreateTablet for tablet 9884157901eb454498ce2b749e1c43c1 (table=client-testtb [id=c065a27ae5044893b5d8ff341586a52a]), range=["", "\x00\x00\x00\t"] I1102 12:55:02.510511 1199 ts_tablet_manager.cc:460] Registered tablet 9884157901eb454498ce2b749e1c43c1 I1102 12:55:02.510565 1198 ts_tablet_manager.cc:460] Registered tablet ce1823b844da44989af4eb1deb03b4b0 I1102 12:55:02.510810 1233 ts_tablet_manager.cc:325] Bootstrapping tablet: 9884157901eb454498ce2b749e1c43c1 I1102 12:55:02.511059 1234 ts_tablet_manager.cc:325] Bootstrapping tablet: ce1823b844da44989af4eb1deb03b4b0 I1102 12:55:02.511677 1234 tablet_bootstrap.cc:234] Tablet ce1823b844da44989af4eb1deb03b4b0: Bootstrap starting. I1102 12:55:02.511682 1233 tablet_bootstrap.cc:234] Tablet 9884157901eb454498ce2b749e1c43c1: Bootstrap starting. I1102 12:55:02.514793 1234 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: ce1823b844da44989af4eb1deb03b4b0 creating new one. I1102 12:55:02.514911 1233 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 9884157901eb454498ce2b749e1c43c1 creating new one. I1102 12:55:02.516137 1234 tablet_bootstrap.cc:234] Tablet ce1823b844da44989af4eb1deb03b4b0: No bootstrap required, opened a new log I1102 12:55:02.516140 1233 tablet_bootstrap.cc:234] Tablet 9884157901eb454498ce2b749e1c43c1: No bootstrap required, opened a new log I1102 12:55:02.516470 1233 ts_tablet_manager.cc:330] Time spent tablet 9884157901eb454498ce2b749e1c43c1 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:02.516640 1234 ts_tablet_manager.cc:330] Time spent tablet ce1823b844da44989af4eb1deb03b4b0 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:02.518014 1233 ts_tablet_manager.cc:350] Time spent starting tablet 9884157901eb454498ce2b749e1c43c1: real 0.001s user 0.001s sys 0.000s I1102 12:55:02.518039 1234 ts_tablet_manager.cc:350] Time spent starting tablet ce1823b844da44989af4eb1deb03b4b0: real 0.001s user 0.001s sys 0.000s I1102 12:55:02.518339 1238 tablet_peer.cc:213] Configuration changed for tablet: 9884157901eb454498ce2b749e1c43c1 in peer: 06fbd87001d34eef866cffdcc74c8ded Changed from: seqno: -1 local: true peers { permanent_uuid: "06fbd87001d34eef866cffdcc74c8ded" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 36221 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "06fbd87001d34eef866cffdcc74c8ded" role: LEADER last_known_addr { host: "127.0.0.1" port: 36221 } } I1102 12:55:02.518486 1237 tablet_peer.cc:213] Configuration changed for tablet: ce1823b844da44989af4eb1deb03b4b0 in peer: 06fbd87001d34eef866cffdcc74c8ded Changed from: seqno: -1 local: true peers { permanent_uuid: "06fbd87001d34eef866cffdcc74c8ded" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 36221 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "06fbd87001d34eef866cffdcc74c8ded" role: LEADER last_known_addr { host: "127.0.0.1" port: 36221 } } I1102 12:55:02.522436 1142 catalog_manager.cc:1127] Tablet: ce1823b844da44989af4eb1deb03b4b0 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "06fbd87001d34eef866cffdcc74c8ded" role: LEADER last_known_addr { host: "127.0.0.1" port: 36221 } } I1102 12:55:02.523881 1142 catalog_manager.cc:1127] Tablet: 9884157901eb454498ce2b749e1c43c1 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "06fbd87001d34eef866cffdcc74c8ded" role: LEADER last_known_addr { host: "127.0.0.1" port: 36221 } } I1102 12:55:02.527209 1149 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:56785: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:02.531445 1201 tablet_service.cc:264] Processing CreateTablet for tablet ea70ecf970114424b2fc23094608da57 (table=client-testtb2 [id=2e0ef8d43f544f4eb9f967234ec64b5d]), range=["", ""] I1102 12:55:02.532393 1201 ts_tablet_manager.cc:460] Registered tablet ea70ecf970114424b2fc23094608da57 I1102 12:55:02.532477 1233 ts_tablet_manager.cc:325] Bootstrapping tablet: ea70ecf970114424b2fc23094608da57 I1102 12:55:02.532877 1233 tablet_bootstrap.cc:234] Tablet ea70ecf970114424b2fc23094608da57: Bootstrap starting. I1102 12:55:02.535051 1233 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: ea70ecf970114424b2fc23094608da57 creating new one. I1102 12:55:02.536049 1233 tablet_bootstrap.cc:234] Tablet ea70ecf970114424b2fc23094608da57: No bootstrap required, opened a new log I1102 12:55:02.536247 1233 ts_tablet_manager.cc:330] Time spent tablet ea70ecf970114424b2fc23094608da57 bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:55:02.537277 1233 ts_tablet_manager.cc:350] Time spent starting tablet ea70ecf970114424b2fc23094608da57: real 0.001s user 0.001s sys 0.000s I1102 12:55:02.537750 1244 tablet_peer.cc:213] Configuration changed for tablet: ea70ecf970114424b2fc23094608da57 in peer: 06fbd87001d34eef866cffdcc74c8ded Changed from: seqno: -1 local: true peers { permanent_uuid: "06fbd87001d34eef866cffdcc74c8ded" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 36221 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "06fbd87001d34eef866cffdcc74c8ded" role: LEADER last_known_addr { host: "127.0.0.1" port: 36221 } } I1102 12:55:02.546556 1147 catalog_manager.cc:1127] Tablet: ea70ecf970114424b2fc23094608da57 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "06fbd87001d34eef866cffdcc74c8ded" role: LEADER last_known_addr { host: "127.0.0.1" port: 36221 } } I1102 12:55:02.554615 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:02.568255 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:02.568356 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:02.568497 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: ea70ecf970114424b2fc23094608da57 I1102 12:55:02.568550 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(ea70ecf970114424b2fc23094608da57) I1102 12:55:02.568584 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(ea70ecf970114424b2fc23094608da57) I1102 12:55:02.568603 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(ea70ecf970114424b2fc23094608da57) I1102 12:55:02.569330 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: ce1823b844da44989af4eb1deb03b4b0 I1102 12:55:02.569365 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(ce1823b844da44989af4eb1deb03b4b0) I1102 12:55:02.569392 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(ce1823b844da44989af4eb1deb03b4b0) I1102 12:55:02.569409 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(ce1823b844da44989af4eb1deb03b4b0) I1102 12:55:02.573920 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 9884157901eb454498ce2b749e1c43c1 I1102 12:55:02.573971 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(9884157901eb454498ce2b749e1c43c1) I1102 12:55:02.573987 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(9884157901eb454498ce2b749e1c43c1) I1102 12:55:02.574000 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(9884157901eb454498ce2b749e1c43c1) I1102 12:55:02.579805 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:02.579836 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:02.579875 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:02.581014 31819 master.cc:124] Master@127.0.0.1:58912 shutting down... I1102 12:55:02.593772 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:02.594672 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:02.595268 31819 master.cc:127] Master@127.0.0.1:58912 shutdown complete. [ OK ] ClientTest.TestEmptyBatch (277 ms) [ RUN ] ClientTest.TestWriteWithDeadMaster I1102 12:55:02.754956 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithDeadMaster.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "4337583d69a94d29a768acdba7ffec64" format_stamp: "Formatted at 2014-11-02 20:55:02 on qiwi" I1102 12:55:02.755095 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithDeadMaster.1414961696528764-31819/minicluster-data/master-0-root uuid: "4337583d69a94d29a768acdba7ffec64" format_stamp: "Formatted at 2014-11-02 20:55:02 on qiwi" I1102 12:55:02.756007 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 494278 I1102 12:55:02.758792 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:02.758808 31819 webserver.cc:117] Document root disabled I1102 12:55:02.759019 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:43947 I1102 12:55:02.760613 1273 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:02.762959 1273 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:02.763887 1273 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:02.765456 1275 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 4337583d69a94d29a768acdba7ffec64 Changed from: seqno: 0 local: true peers { permanent_uuid: "4337583d69a94d29a768acdba7ffec64" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "4337583d69a94d29a768acdba7ffec64" role: LEADER } I1102 12:55:02.765517 1275 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "4337583d69a94d29a768acdba7ffec64" role: LEADER } I1102 12:55:02.765964 1273 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:02.768082 1273 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:02.769027 1273 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:02.770162 1273 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:02.770588 1279 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 4337583d69a94d29a768acdba7ffec64 Changed from: seqno: 0 local: true peers { permanent_uuid: "4337583d69a94d29a768acdba7ffec64" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "4337583d69a94d29a768acdba7ffec64" role: LEADER } I1102 12:55:02.770639 1273 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:02.770644 1279 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "4337583d69a94d29a768acdba7ffec64" role: LEADER } I1102 12:55:02.785171 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithDeadMaster.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "da4f315abc2f4039b8f0c9c865efab84" format_stamp: "Formatted at 2014-11-02 20:55:02 on qiwi" I1102 12:55:02.785315 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithDeadMaster.1414961696528764-31819/minicluster-data/ts-0-root uuid: "da4f315abc2f4039b8f0c9c865efab84" format_stamp: "Formatted at 2014-11-02 20:55:02 on qiwi" I1102 12:55:02.786229 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 494278 I1102 12:55:02.790727 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:02.790745 31819 webserver.cc:117] Document root disabled I1102 12:55:02.790984 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:43202 I1102 12:55:02.791116 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:02.792727 1322 heartbeater.cc:227] Registering TS with master... I1102 12:55:02.792795 1322 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:02.793233 1251 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "da4f315abc2f4039b8f0c9c865efab84" instance_seqno: 1414961702790451 } with Master I1102 12:55:02.793489 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00219388s I1102 12:55:02.798497 1254 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:57989: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:02.807086 1300 tablet_service.cc:264] Processing CreateTablet for tablet d6912951b1d246c99da541e13dfde54d (table=client-testtb [id=310015056a654d25beb229b2982cdacb]), range=["", "\x00\x00\x00\t"] I1102 12:55:02.807201 1301 tablet_service.cc:264] Processing CreateTablet for tablet 1b8a94dbb4d64a5aade4602d19052887 (table=client-testtb [id=310015056a654d25beb229b2982cdacb]), range=["\x00\x00\x00\t", ""] I1102 12:55:02.808544 1300 ts_tablet_manager.cc:460] Registered tablet d6912951b1d246c99da541e13dfde54d I1102 12:55:02.808584 1301 ts_tablet_manager.cc:460] Registered tablet 1b8a94dbb4d64a5aade4602d19052887 I1102 12:55:02.809334 1332 ts_tablet_manager.cc:325] Bootstrapping tablet: 1b8a94dbb4d64a5aade4602d19052887 I1102 12:55:02.809332 1331 ts_tablet_manager.cc:325] Bootstrapping tablet: d6912951b1d246c99da541e13dfde54d I1102 12:55:02.810246 1331 tablet_bootstrap.cc:234] Tablet d6912951b1d246c99da541e13dfde54d: Bootstrap starting. I1102 12:55:02.810261 1332 tablet_bootstrap.cc:234] Tablet 1b8a94dbb4d64a5aade4602d19052887: Bootstrap starting. I1102 12:55:02.814298 1332 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 1b8a94dbb4d64a5aade4602d19052887 creating new one. I1102 12:55:02.814769 1331 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: d6912951b1d246c99da541e13dfde54d creating new one. I1102 12:55:02.816547 1332 tablet_bootstrap.cc:234] Tablet 1b8a94dbb4d64a5aade4602d19052887: No bootstrap required, opened a new log I1102 12:55:02.816706 1331 tablet_bootstrap.cc:234] Tablet d6912951b1d246c99da541e13dfde54d: No bootstrap required, opened a new log I1102 12:55:02.817069 1332 ts_tablet_manager.cc:330] Time spent tablet 1b8a94dbb4d64a5aade4602d19052887 bootstrap: real 0.008s user 0.005s sys 0.001s I1102 12:55:02.817183 1331 ts_tablet_manager.cc:330] Time spent tablet d6912951b1d246c99da541e13dfde54d bootstrap: real 0.008s user 0.006s sys 0.002s I1102 12:55:02.818364 1331 ts_tablet_manager.cc:350] Time spent starting tablet d6912951b1d246c99da541e13dfde54d: real 0.001s user 0.001s sys 0.000s I1102 12:55:02.818425 1332 ts_tablet_manager.cc:350] Time spent starting tablet 1b8a94dbb4d64a5aade4602d19052887: real 0.001s user 0.001s sys 0.000s I1102 12:55:02.818773 1336 tablet_peer.cc:213] Configuration changed for tablet: d6912951b1d246c99da541e13dfde54d in peer: da4f315abc2f4039b8f0c9c865efab84 Changed from: seqno: -1 local: true peers { permanent_uuid: "da4f315abc2f4039b8f0c9c865efab84" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 48340 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "da4f315abc2f4039b8f0c9c865efab84" role: LEADER last_known_addr { host: "127.0.0.1" port: 48340 } } I1102 12:55:02.818930 1335 tablet_peer.cc:213] Configuration changed for tablet: 1b8a94dbb4d64a5aade4602d19052887 in peer: da4f315abc2f4039b8f0c9c865efab84 Changed from: seqno: -1 local: true peers { permanent_uuid: "da4f315abc2f4039b8f0c9c865efab84" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 48340 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "da4f315abc2f4039b8f0c9c865efab84" role: LEADER last_known_addr { host: "127.0.0.1" port: 48340 } } I1102 12:55:02.826339 1254 catalog_manager.cc:1127] Tablet: d6912951b1d246c99da541e13dfde54d reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "da4f315abc2f4039b8f0c9c865efab84" role: LEADER last_known_addr { host: "127.0.0.1" port: 48340 } } I1102 12:55:02.827847 1254 catalog_manager.cc:1127] Tablet: 1b8a94dbb4d64a5aade4602d19052887 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "da4f315abc2f4039b8f0c9c865efab84" role: LEADER last_known_addr { host: "127.0.0.1" port: 48340 } } I1102 12:55:02.837311 1259 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:57989: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:02.841819 1302 tablet_service.cc:264] Processing CreateTablet for tablet 817f1c1583ed41bb841d55ca0152167e (table=client-testtb2 [id=3babad7cc0c44b1c98624be5a4554e28]), range=["", ""] I1102 12:55:02.842805 1302 ts_tablet_manager.cc:460] Registered tablet 817f1c1583ed41bb841d55ca0152167e I1102 12:55:02.842883 1331 ts_tablet_manager.cc:325] Bootstrapping tablet: 817f1c1583ed41bb841d55ca0152167e I1102 12:55:02.843262 1331 tablet_bootstrap.cc:234] Tablet 817f1c1583ed41bb841d55ca0152167e: Bootstrap starting. I1102 12:55:02.845867 1331 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 817f1c1583ed41bb841d55ca0152167e creating new one. I1102 12:55:02.847782 1331 tablet_bootstrap.cc:234] Tablet 817f1c1583ed41bb841d55ca0152167e: No bootstrap required, opened a new log I1102 12:55:02.848285 1331 ts_tablet_manager.cc:330] Time spent tablet 817f1c1583ed41bb841d55ca0152167e bootstrap: real 0.005s user 0.004s sys 0.001s I1102 12:55:02.850319 1331 ts_tablet_manager.cc:350] Time spent starting tablet 817f1c1583ed41bb841d55ca0152167e: real 0.002s user 0.001s sys 0.000s I1102 12:55:02.850919 1342 tablet_peer.cc:213] Configuration changed for tablet: 817f1c1583ed41bb841d55ca0152167e in peer: da4f315abc2f4039b8f0c9c865efab84 Changed from: seqno: -1 local: true peers { permanent_uuid: "da4f315abc2f4039b8f0c9c865efab84" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 48340 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "da4f315abc2f4039b8f0c9c865efab84" role: LEADER last_known_addr { host: "127.0.0.1" port: 48340 } } I1102 12:55:02.862316 1250 catalog_manager.cc:1127] Tablet: 817f1c1583ed41bb841d55ca0152167e reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "da4f315abc2f4039b8f0c9c865efab84" role: LEADER last_known_addr { host: "127.0.0.1" port: 48340 } } I1102 12:55:02.869288 31819 master.cc:124] Master@127.0.0.1:34389 shutting down... I1102 12:55:02.882853 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:02.883930 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:02.884871 31819 master.cc:127] Master@127.0.0.1:34389 shutdown complete. I1102 12:55:02.887675 1247 messenger.cc:190] Service unavailable: RPC Service is not registered: Call TSHeartbeat from 127.0.0.1:57988 (request call id 10) W1102 12:55:02.888084 1322 heartbeater.cc:277] Failed to heartbeat: Remote error: Failed to send heartbeat: Service unavailable: RPC Service is not registered: Call TSHeartbeat from 127.0.0.1:57988 (request call id 10) W1102 12:55:02.892329 1330 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:34389: connect: Connection refused (error 111) W1102 12:55:02.892472 1327 meta_cache.cc:394] GetTableLocations(client-testtb, ) failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:34389: connect: Connection refused (error 111) I1102 12:55:02.892921 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:02.910204 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:02.910370 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:02.910492 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: d6912951b1d246c99da541e13dfde54d I1102 12:55:02.910598 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(d6912951b1d246c99da541e13dfde54d) I1102 12:55:02.910677 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(d6912951b1d246c99da541e13dfde54d) I1102 12:55:02.910732 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(d6912951b1d246c99da541e13dfde54d) I1102 12:55:02.912211 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 1b8a94dbb4d64a5aade4602d19052887 I1102 12:55:02.912276 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(1b8a94dbb4d64a5aade4602d19052887) I1102 12:55:02.912308 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(1b8a94dbb4d64a5aade4602d19052887) I1102 12:55:02.912333 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(1b8a94dbb4d64a5aade4602d19052887) I1102 12:55:02.913466 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 817f1c1583ed41bb841d55ca0152167e I1102 12:55:02.913511 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(817f1c1583ed41bb841d55ca0152167e) I1102 12:55:02.913535 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(817f1c1583ed41bb841d55ca0152167e) I1102 12:55:02.913553 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(817f1c1583ed41bb841d55ca0152167e) I1102 12:55:02.926087 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:02.926110 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:02.926137 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! [ OK ] ClientTest.TestWriteWithDeadMaster (327 ms) [ RUN ] ClientTest.TestWriteWithDeadTabletServer I1102 12:55:03.068976 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithDeadTabletServer.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "2b0d4a34059c4bbf9de1bd594959a337" format_stamp: "Formatted at 2014-11-02 20:55:02 on qiwi" I1102 12:55:03.069077 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithDeadTabletServer.1414961696528764-31819/minicluster-data/master-0-root uuid: "2b0d4a34059c4bbf9de1bd594959a337" format_stamp: "Formatted at 2014-11-02 20:55:02 on qiwi" I1102 12:55:03.069919 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 494778 I1102 12:55:03.072552 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:03.072567 31819 webserver.cc:117] Document root disabled I1102 12:55:03.072759 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:52837 I1102 12:55:03.073978 1370 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:03.075742 1370 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:03.076580 1370 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:03.077841 1372 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 2b0d4a34059c4bbf9de1bd594959a337 Changed from: seqno: 0 local: true peers { permanent_uuid: "2b0d4a34059c4bbf9de1bd594959a337" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "2b0d4a34059c4bbf9de1bd594959a337" role: LEADER } I1102 12:55:03.077894 1372 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "2b0d4a34059c4bbf9de1bd594959a337" role: LEADER } I1102 12:55:03.078356 1370 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:03.080080 1370 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:03.080816 1370 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:03.081802 1370 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:03.082109 1376 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 2b0d4a34059c4bbf9de1bd594959a337 Changed from: seqno: 0 local: true peers { permanent_uuid: "2b0d4a34059c4bbf9de1bd594959a337" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "2b0d4a34059c4bbf9de1bd594959a337" role: LEADER } I1102 12:55:03.082156 1370 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:03.082154 1376 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "2b0d4a34059c4bbf9de1bd594959a337" role: LEADER } I1102 12:55:03.093010 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithDeadTabletServer.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "d9a259058cfc4adf992c8198ca0bdbff" format_stamp: "Formatted at 2014-11-02 20:55:03 on qiwi" I1102 12:55:03.093127 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithDeadTabletServer.1414961696528764-31819/minicluster-data/ts-0-root uuid: "d9a259058cfc4adf992c8198ca0bdbff" format_stamp: "Formatted at 2014-11-02 20:55:03 on qiwi" I1102 12:55:03.093925 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 494778 I1102 12:55:03.099566 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:03.099609 31819 webserver.cc:117] Document root disabled I1102 12:55:03.100144 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:34913 I1102 12:55:03.100463 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:03.103875 1419 heartbeater.cc:227] Registering TS with master... I1102 12:55:03.103996 1419 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:03.104833 1349 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "d9a259058cfc4adf992c8198ca0bdbff" instance_seqno: 1414961703098959 } with Master I1102 12:55:03.105342 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00446245s I1102 12:55:03.112040 1351 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:58617: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:03.123883 1397 tablet_service.cc:264] Processing CreateTablet for tablet 2302fe161a9546ec8ad49088e093c7d8 (table=client-testtb [id=16ae22274fb14278b6709fb42ec86f1f]), range=["\x00\x00\x00\t", ""] I1102 12:55:03.123981 1398 tablet_service.cc:264] Processing CreateTablet for tablet 9f2eea8f88064fb1836e61f91125543f (table=client-testtb [id=16ae22274fb14278b6709fb42ec86f1f]), range=["", "\x00\x00\x00\t"] I1102 12:55:03.125823 1397 ts_tablet_manager.cc:460] Registered tablet 2302fe161a9546ec8ad49088e093c7d8 I1102 12:55:03.125870 1398 ts_tablet_manager.cc:460] Registered tablet 9f2eea8f88064fb1836e61f91125543f I1102 12:55:03.126269 1428 ts_tablet_manager.cc:325] Bootstrapping tablet: 2302fe161a9546ec8ad49088e093c7d8 I1102 12:55:03.126711 1429 ts_tablet_manager.cc:325] Bootstrapping tablet: 9f2eea8f88064fb1836e61f91125543f I1102 12:55:03.127298 1428 tablet_bootstrap.cc:234] Tablet 2302fe161a9546ec8ad49088e093c7d8: Bootstrap starting. I1102 12:55:03.127604 1429 tablet_bootstrap.cc:234] Tablet 9f2eea8f88064fb1836e61f91125543f: Bootstrap starting. I1102 12:55:03.131047 1428 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 2302fe161a9546ec8ad49088e093c7d8 creating new one. I1102 12:55:03.131153 1429 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 9f2eea8f88064fb1836e61f91125543f creating new one. I1102 12:55:03.132666 1428 tablet_bootstrap.cc:234] Tablet 2302fe161a9546ec8ad49088e093c7d8: No bootstrap required, opened a new log I1102 12:55:03.132735 1429 tablet_bootstrap.cc:234] Tablet 9f2eea8f88064fb1836e61f91125543f: No bootstrap required, opened a new log I1102 12:55:03.132971 1428 ts_tablet_manager.cc:330] Time spent tablet 2302fe161a9546ec8ad49088e093c7d8 bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:55:03.133018 1429 ts_tablet_manager.cc:330] Time spent tablet 9f2eea8f88064fb1836e61f91125543f bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:03.134802 1429 ts_tablet_manager.cc:350] Time spent starting tablet 9f2eea8f88064fb1836e61f91125543f: real 0.002s user 0.001s sys 0.001s I1102 12:55:03.134840 1428 ts_tablet_manager.cc:350] Time spent starting tablet 2302fe161a9546ec8ad49088e093c7d8: real 0.002s user 0.001s sys 0.001s I1102 12:55:03.135597 1432 tablet_peer.cc:213] Configuration changed for tablet: 2302fe161a9546ec8ad49088e093c7d8 in peer: d9a259058cfc4adf992c8198ca0bdbff Changed from: seqno: -1 local: true peers { permanent_uuid: "d9a259058cfc4adf992c8198ca0bdbff" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 40474 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "d9a259058cfc4adf992c8198ca0bdbff" role: LEADER last_known_addr { host: "127.0.0.1" port: 40474 } } I1102 12:55:03.135602 1433 tablet_peer.cc:213] Configuration changed for tablet: 9f2eea8f88064fb1836e61f91125543f in peer: d9a259058cfc4adf992c8198ca0bdbff Changed from: seqno: -1 local: true peers { permanent_uuid: "d9a259058cfc4adf992c8198ca0bdbff" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 40474 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "d9a259058cfc4adf992c8198ca0bdbff" role: LEADER last_known_addr { host: "127.0.0.1" port: 40474 } } I1102 12:55:03.138782 1350 catalog_manager.cc:1127] Tablet: 2302fe161a9546ec8ad49088e093c7d8 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "d9a259058cfc4adf992c8198ca0bdbff" role: LEADER last_known_addr { host: "127.0.0.1" port: 40474 } } I1102 12:55:03.140842 1350 catalog_manager.cc:1127] Tablet: 9f2eea8f88064fb1836e61f91125543f reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "d9a259058cfc4adf992c8198ca0bdbff" role: LEADER last_known_addr { host: "127.0.0.1" port: 40474 } } I1102 12:55:03.147204 1355 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:58617: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:03.153180 1399 tablet_service.cc:264] Processing CreateTablet for tablet 552370936fab4598a9af344de7b32925 (table=client-testtb2 [id=05acd984431e43a681dcd68da78cd057]), range=["", ""] I1102 12:55:03.154505 1399 ts_tablet_manager.cc:460] Registered tablet 552370936fab4598a9af344de7b32925 I1102 12:55:03.154625 1429 ts_tablet_manager.cc:325] Bootstrapping tablet: 552370936fab4598a9af344de7b32925 I1102 12:55:03.155192 1429 tablet_bootstrap.cc:234] Tablet 552370936fab4598a9af344de7b32925: Bootstrap starting. I1102 12:55:03.159618 1429 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 552370936fab4598a9af344de7b32925 creating new one. I1102 12:55:03.161772 1429 tablet_bootstrap.cc:234] Tablet 552370936fab4598a9af344de7b32925: No bootstrap required, opened a new log I1102 12:55:03.162431 1429 ts_tablet_manager.cc:330] Time spent tablet 552370936fab4598a9af344de7b32925 bootstrap: real 0.008s user 0.005s sys 0.001s I1102 12:55:03.164768 1429 ts_tablet_manager.cc:350] Time spent starting tablet 552370936fab4598a9af344de7b32925: real 0.002s user 0.001s sys 0.000s I1102 12:55:03.165503 1439 tablet_peer.cc:213] Configuration changed for tablet: 552370936fab4598a9af344de7b32925 in peer: d9a259058cfc4adf992c8198ca0bdbff Changed from: seqno: -1 local: true peers { permanent_uuid: "d9a259058cfc4adf992c8198ca0bdbff" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 40474 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "d9a259058cfc4adf992c8198ca0bdbff" role: LEADER last_known_addr { host: "127.0.0.1" port: 40474 } } I1102 12:55:03.176669 1355 catalog_manager.cc:1127] Tablet: 552370936fab4598a9af344de7b32925 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "d9a259058cfc4adf992c8198ca0bdbff" role: LEADER last_known_addr { host: "127.0.0.1" port: 40474 } } I1102 12:55:03.189306 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:03.207865 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:03.208096 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:03.208235 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 552370936fab4598a9af344de7b32925 I1102 12:55:03.208328 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(552370936fab4598a9af344de7b32925) I1102 12:55:03.208387 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(552370936fab4598a9af344de7b32925) I1102 12:55:03.208441 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(552370936fab4598a9af344de7b32925) I1102 12:55:03.210132 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 2302fe161a9546ec8ad49088e093c7d8 I1102 12:55:03.210216 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(2302fe161a9546ec8ad49088e093c7d8) I1102 12:55:03.210259 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(2302fe161a9546ec8ad49088e093c7d8) I1102 12:55:03.210293 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(2302fe161a9546ec8ad49088e093c7d8) I1102 12:55:03.212303 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 9f2eea8f88064fb1836e61f91125543f I1102 12:55:03.212400 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(9f2eea8f88064fb1836e61f91125543f) I1102 12:55:03.212448 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(9f2eea8f88064fb1836e61f91125543f) I1102 12:55:03.212481 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(9f2eea8f88064fb1836e61f91125543f) I1102 12:55:03.225256 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:03.225276 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:03.225306 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! W1102 12:55:03.227665 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.227737 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.240039 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.240097 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.254487 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.254549 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.272923 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.272981 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.299135 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.299378 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.329221 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.329458 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.365970 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.366227 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.404556 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.404796 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.448518 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.448757 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.495009 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.495249 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.538781 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.538879 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.592839 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.592923 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.649863 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.649943 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.706374 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.706449 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.771299 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.771543 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.840600 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.840749 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.914489 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.914703 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.992169 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:03.992415 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:04.075151 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:04.075397 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:04.162470 1427 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:04.162549 1424 meta_cache.cc:159] Replica 9f2eea8f88064fb1836e61f91125543f on ts d9a259058cfc4adf992c8198ca0bdbff has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:40474: connect: Connection refused (error 111) W1102 12:55:04.251232 1424 batcher.cc:388] Write(9f2eea8f88064fb1836e61f91125543f, 1 ops) failed: Timed out: Write(9f2eea8f88064fb1836e61f91125543f, 1 ops) timed out I1102 12:55:04.251785 31819 master.cc:124] Master@127.0.0.1:43627 shutting down... I1102 12:55:04.265924 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:04.266681 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:04.267246 31819 master.cc:127] Master@127.0.0.1:43627 shutdown complete. [ OK ] ClientTest.TestWriteWithDeadTabletServer (1349 ms) [ RUN ] ClientTest.TestApplyToSessionWithoutFlushing_OpsInFlight I1102 12:55:04.430881 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestApplyToSessionWithoutFlushing_OpsInFlight.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "4cc329cb0fa64ab0aa20e99f964740ba" format_stamp: "Formatted at 2014-11-02 20:55:04 on qiwi" I1102 12:55:04.430986 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestApplyToSessionWithoutFlushing_OpsInFlight.1414961696528764-31819/minicluster-data/master-0-root uuid: "4cc329cb0fa64ab0aa20e99f964740ba" format_stamp: "Formatted at 2014-11-02 20:55:04 on qiwi" I1102 12:55:04.431831 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 495278 I1102 12:55:04.434620 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:04.434636 31819 webserver.cc:117] Document root disabled I1102 12:55:04.434845 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:59479 I1102 12:55:04.436218 1480 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:04.438465 1480 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:04.439391 1480 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:04.441001 1482 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 4cc329cb0fa64ab0aa20e99f964740ba Changed from: seqno: 0 local: true peers { permanent_uuid: "4cc329cb0fa64ab0aa20e99f964740ba" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "4cc329cb0fa64ab0aa20e99f964740ba" role: LEADER } I1102 12:55:04.441063 1482 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "4cc329cb0fa64ab0aa20e99f964740ba" role: LEADER } I1102 12:55:04.441555 1480 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:04.443814 1480 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:04.444736 1480 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:04.445929 1480 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:04.446310 1486 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 4cc329cb0fa64ab0aa20e99f964740ba Changed from: seqno: 0 local: true peers { permanent_uuid: "4cc329cb0fa64ab0aa20e99f964740ba" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "4cc329cb0fa64ab0aa20e99f964740ba" role: LEADER } I1102 12:55:04.446382 1480 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:04.446385 1486 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "4cc329cb0fa64ab0aa20e99f964740ba" role: LEADER } I1102 12:55:04.460837 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestApplyToSessionWithoutFlushing_OpsInFlight.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "731411e1d10144719c66062858cfba2e" format_stamp: "Formatted at 2014-11-02 20:55:04 on qiwi" I1102 12:55:04.460966 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestApplyToSessionWithoutFlushing_OpsInFlight.1414961696528764-31819/minicluster-data/ts-0-root uuid: "731411e1d10144719c66062858cfba2e" format_stamp: "Formatted at 2014-11-02 20:55:04 on qiwi" I1102 12:55:04.463129 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 495278 I1102 12:55:04.469074 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:04.469094 31819 webserver.cc:117] Document root disabled I1102 12:55:04.469375 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:43954 I1102 12:55:04.469516 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:04.471321 1529 heartbeater.cc:227] Registering TS with master... I1102 12:55:04.471386 1529 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:04.471804 1458 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "731411e1d10144719c66062858cfba2e" instance_seqno: 1414961704468768 } with Master I1102 12:55:04.471899 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00225595s I1102 12:55:04.475445 1460 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:59605: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:04.481976 1507 tablet_service.cc:264] Processing CreateTablet for tablet 93e35f573f034e2fbe320af76f28b6eb (table=client-testtb [id=3ecb223977bb4a51a415be85e521a416]), range=["\x00\x00\x00\t", ""] I1102 12:55:04.482034 1508 tablet_service.cc:264] Processing CreateTablet for tablet 23a3a305a236458597c77c74206c85a3 (table=client-testtb [id=3ecb223977bb4a51a415be85e521a416]), range=["", "\x00\x00\x00\t"] I1102 12:55:04.483114 1507 ts_tablet_manager.cc:460] Registered tablet 93e35f573f034e2fbe320af76f28b6eb I1102 12:55:04.483206 1508 ts_tablet_manager.cc:460] Registered tablet 23a3a305a236458597c77c74206c85a3 I1102 12:55:04.483597 1540 ts_tablet_manager.cc:325] Bootstrapping tablet: 93e35f573f034e2fbe320af76f28b6eb I1102 12:55:04.483623 1539 ts_tablet_manager.cc:325] Bootstrapping tablet: 23a3a305a236458597c77c74206c85a3 I1102 12:55:04.484083 1539 tablet_bootstrap.cc:234] Tablet 23a3a305a236458597c77c74206c85a3: Bootstrap starting. I1102 12:55:04.484099 1540 tablet_bootstrap.cc:234] Tablet 93e35f573f034e2fbe320af76f28b6eb: Bootstrap starting. I1102 12:55:04.486728 1539 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 23a3a305a236458597c77c74206c85a3 creating new one. I1102 12:55:04.486763 1540 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 93e35f573f034e2fbe320af76f28b6eb creating new one. I1102 12:55:04.488199 1540 tablet_bootstrap.cc:234] Tablet 93e35f573f034e2fbe320af76f28b6eb: No bootstrap required, opened a new log I1102 12:55:04.488210 1539 tablet_bootstrap.cc:234] Tablet 23a3a305a236458597c77c74206c85a3: No bootstrap required, opened a new log I1102 12:55:04.488446 1539 ts_tablet_manager.cc:330] Time spent tablet 23a3a305a236458597c77c74206c85a3 bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:04.488450 1540 ts_tablet_manager.cc:330] Time spent tablet 93e35f573f034e2fbe320af76f28b6eb bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:04.489879 1540 ts_tablet_manager.cc:350] Time spent starting tablet 93e35f573f034e2fbe320af76f28b6eb: real 0.001s user 0.001s sys 0.000s I1102 12:55:04.489893 1539 ts_tablet_manager.cc:350] Time spent starting tablet 23a3a305a236458597c77c74206c85a3: real 0.001s user 0.001s sys 0.000s I1102 12:55:04.490633 1544 tablet_peer.cc:213] Configuration changed for tablet: 23a3a305a236458597c77c74206c85a3 in peer: 731411e1d10144719c66062858cfba2e Changed from: seqno: -1 local: true peers { permanent_uuid: "731411e1d10144719c66062858cfba2e" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 32837 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "731411e1d10144719c66062858cfba2e" role: LEADER last_known_addr { host: "127.0.0.1" port: 32837 } } I1102 12:55:04.490631 1543 tablet_peer.cc:213] Configuration changed for tablet: 93e35f573f034e2fbe320af76f28b6eb in peer: 731411e1d10144719c66062858cfba2e Changed from: seqno: -1 local: true peers { permanent_uuid: "731411e1d10144719c66062858cfba2e" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 32837 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "731411e1d10144719c66062858cfba2e" role: LEADER last_known_addr { host: "127.0.0.1" port: 32837 } } I1102 12:55:04.493612 1458 catalog_manager.cc:1127] Tablet: 93e35f573f034e2fbe320af76f28b6eb reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "731411e1d10144719c66062858cfba2e" role: LEADER last_known_addr { host: "127.0.0.1" port: 32837 } } I1102 12:55:04.495242 1458 catalog_manager.cc:1127] Tablet: 23a3a305a236458597c77c74206c85a3 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "731411e1d10144719c66062858cfba2e" role: LEADER last_known_addr { host: "127.0.0.1" port: 32837 } } I1102 12:55:04.499392 1463 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:59605: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:04.504133 1509 tablet_service.cc:264] Processing CreateTablet for tablet 8e466d4003d440a4b90dda3530a78960 (table=client-testtb2 [id=e1214342344d40319fff454cd5561328]), range=["", ""] I1102 12:55:04.505180 1509 ts_tablet_manager.cc:460] Registered tablet 8e466d4003d440a4b90dda3530a78960 I1102 12:55:04.505295 1540 ts_tablet_manager.cc:325] Bootstrapping tablet: 8e466d4003d440a4b90dda3530a78960 I1102 12:55:04.505763 1540 tablet_bootstrap.cc:234] Tablet 8e466d4003d440a4b90dda3530a78960: Bootstrap starting. I1102 12:55:04.508357 1540 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 8e466d4003d440a4b90dda3530a78960 creating new one. I1102 12:55:04.509804 1540 tablet_bootstrap.cc:234] Tablet 8e466d4003d440a4b90dda3530a78960: No bootstrap required, opened a new log I1102 12:55:04.510236 1540 ts_tablet_manager.cc:330] Time spent tablet 8e466d4003d440a4b90dda3530a78960 bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:04.512078 1540 ts_tablet_manager.cc:350] Time spent starting tablet 8e466d4003d440a4b90dda3530a78960: real 0.002s user 0.001s sys 0.000s I1102 12:55:04.512570 1550 tablet_peer.cc:213] Configuration changed for tablet: 8e466d4003d440a4b90dda3530a78960 in peer: 731411e1d10144719c66062858cfba2e Changed from: seqno: -1 local: true peers { permanent_uuid: "731411e1d10144719c66062858cfba2e" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 32837 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "731411e1d10144719c66062858cfba2e" role: LEADER last_known_addr { host: "127.0.0.1" port: 32837 } } I1102 12:55:04.518661 1460 catalog_manager.cc:1127] Tablet: 8e466d4003d440a4b90dda3530a78960 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "731411e1d10144719c66062858cfba2e" role: LEADER last_known_addr { host: "127.0.0.1" port: 32837 } } W1102 12:55:04.524971 31819 client.cc:427] Closing Session with pending operations. I1102 12:55:04.536631 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:04.554267 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:04.554467 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:04.554811 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 8e466d4003d440a4b90dda3530a78960 I1102 12:55:04.554911 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(8e466d4003d440a4b90dda3530a78960) I1102 12:55:04.554980 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(8e466d4003d440a4b90dda3530a78960) I1102 12:55:04.555016 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(8e466d4003d440a4b90dda3530a78960) I1102 12:55:04.556318 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 93e35f573f034e2fbe320af76f28b6eb I1102 12:55:04.556388 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(93e35f573f034e2fbe320af76f28b6eb) I1102 12:55:04.563282 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(93e35f573f034e2fbe320af76f28b6eb) I1102 12:55:04.563313 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(93e35f573f034e2fbe320af76f28b6eb) I1102 12:55:04.564378 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 23a3a305a236458597c77c74206c85a3 I1102 12:55:04.564424 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(23a3a305a236458597c77c74206c85a3) I1102 12:55:04.564447 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(23a3a305a236458597c77c74206c85a3) I1102 12:55:04.564465 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(23a3a305a236458597c77c74206c85a3) I1102 12:55:04.570823 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:04.570839 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:04.570866 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:04.571632 31819 master.cc:124] Master@127.0.0.1:51344 shutting down... I1102 12:55:04.583287 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:04.583739 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:04.584112 31819 master.cc:127] Master@127.0.0.1:51344 shutdown complete. [ OK ] ClientTest.TestApplyToSessionWithoutFlushing_OpsInFlight (311 ms) [ RUN ] ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered I1102 12:55:04.730007 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "4c10185009f04208a6e891f45eff63f2" format_stamp: "Formatted at 2014-11-02 20:55:04 on qiwi" I1102 12:55:04.730110 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1414961696528764-31819/minicluster-data/master-0-root uuid: "4c10185009f04208a6e891f45eff63f2" format_stamp: "Formatted at 2014-11-02 20:55:04 on qiwi" I1102 12:55:04.730948 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 495278 I1102 12:55:04.733659 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:04.733675 31819 webserver.cc:117] Document root disabled I1102 12:55:04.733877 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:52058 I1102 12:55:04.735149 1579 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:04.737184 1579 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:04.738026 1579 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:04.739398 1581 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 4c10185009f04208a6e891f45eff63f2 Changed from: seqno: 0 local: true peers { permanent_uuid: "4c10185009f04208a6e891f45eff63f2" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "4c10185009f04208a6e891f45eff63f2" role: LEADER } I1102 12:55:04.739454 1581 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "4c10185009f04208a6e891f45eff63f2" role: LEADER } I1102 12:55:04.739915 1579 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:04.741914 1579 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:04.742739 1579 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:04.743737 1579 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:04.744103 1585 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 4c10185009f04208a6e891f45eff63f2 Changed from: seqno: 0 local: true peers { permanent_uuid: "4c10185009f04208a6e891f45eff63f2" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "4c10185009f04208a6e891f45eff63f2" role: LEADER } I1102 12:55:04.744153 1579 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:04.744158 1585 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "4c10185009f04208a6e891f45eff63f2" role: LEADER } I1102 12:55:04.756170 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "2a0fe46c06ed4be8bcb0c977eb160888" format_stamp: "Formatted at 2014-11-02 20:55:04 on qiwi" I1102 12:55:04.756259 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1414961696528764-31819/minicluster-data/ts-0-root uuid: "2a0fe46c06ed4be8bcb0c977eb160888" format_stamp: "Formatted at 2014-11-02 20:55:04 on qiwi" I1102 12:55:04.757112 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 495278 I1102 12:55:04.761309 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:04.761327 31819 webserver.cc:117] Document root disabled I1102 12:55:04.761544 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:57636 I1102 12:55:04.761668 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:04.763043 1629 heartbeater.cc:227] Registering TS with master... I1102 12:55:04.763092 1629 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:04.763401 1558 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "2a0fe46c06ed4be8bcb0c977eb160888" instance_seqno: 1414961704761043 } with Master I1102 12:55:04.764015 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00217418s I1102 12:55:04.766856 1560 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:55279: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:04.772100 1606 tablet_service.cc:264] Processing CreateTablet for tablet 33c9287806eb49b993d596ec809738c1 (table=client-testtb [id=6093beae73b740358bbad61b8b675a6a]), range=["", "\x00\x00\x00\t"] I1102 12:55:04.772146 1608 tablet_service.cc:264] Processing CreateTablet for tablet 11710bcc05da4accb284fefc5032d8f6 (table=client-testtb [id=6093beae73b740358bbad61b8b675a6a]), range=["\x00\x00\x00\t", ""] I1102 12:55:04.772951 1606 ts_tablet_manager.cc:460] Registered tablet 33c9287806eb49b993d596ec809738c1 I1102 12:55:04.772995 1608 ts_tablet_manager.cc:460] Registered tablet 11710bcc05da4accb284fefc5032d8f6 I1102 12:55:04.773397 1638 ts_tablet_manager.cc:325] Bootstrapping tablet: 33c9287806eb49b993d596ec809738c1 I1102 12:55:04.773416 1639 ts_tablet_manager.cc:325] Bootstrapping tablet: 11710bcc05da4accb284fefc5032d8f6 I1102 12:55:04.773784 1639 tablet_bootstrap.cc:234] Tablet 11710bcc05da4accb284fefc5032d8f6: Bootstrap starting. I1102 12:55:04.773798 1638 tablet_bootstrap.cc:234] Tablet 33c9287806eb49b993d596ec809738c1: Bootstrap starting. I1102 12:55:04.775813 1639 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11710bcc05da4accb284fefc5032d8f6 creating new one. I1102 12:55:04.775842 1638 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 33c9287806eb49b993d596ec809738c1 creating new one. I1102 12:55:04.777287 1638 tablet_bootstrap.cc:234] Tablet 33c9287806eb49b993d596ec809738c1: No bootstrap required, opened a new log I1102 12:55:04.777287 1639 tablet_bootstrap.cc:234] Tablet 11710bcc05da4accb284fefc5032d8f6: No bootstrap required, opened a new log I1102 12:55:04.777478 1639 ts_tablet_manager.cc:330] Time spent tablet 11710bcc05da4accb284fefc5032d8f6 bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:55:04.777478 1638 ts_tablet_manager.cc:330] Time spent tablet 33c9287806eb49b993d596ec809738c1 bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:55:04.778625 1638 ts_tablet_manager.cc:350] Time spent starting tablet 33c9287806eb49b993d596ec809738c1: real 0.001s user 0.001s sys 0.000s I1102 12:55:04.778694 1639 ts_tablet_manager.cc:350] Time spent starting tablet 11710bcc05da4accb284fefc5032d8f6: real 0.001s user 0.001s sys 0.000s I1102 12:55:04.778970 1642 tablet_peer.cc:213] Configuration changed for tablet: 33c9287806eb49b993d596ec809738c1 in peer: 2a0fe46c06ed4be8bcb0c977eb160888 Changed from: seqno: -1 local: true peers { permanent_uuid: "2a0fe46c06ed4be8bcb0c977eb160888" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 40525 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "2a0fe46c06ed4be8bcb0c977eb160888" role: LEADER last_known_addr { host: "127.0.0.1" port: 40525 } } I1102 12:55:04.779120 1643 tablet_peer.cc:213] Configuration changed for tablet: 11710bcc05da4accb284fefc5032d8f6 in peer: 2a0fe46c06ed4be8bcb0c977eb160888 Changed from: seqno: -1 local: true peers { permanent_uuid: "2a0fe46c06ed4be8bcb0c977eb160888" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 40525 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "2a0fe46c06ed4be8bcb0c977eb160888" role: LEADER last_known_addr { host: "127.0.0.1" port: 40525 } } I1102 12:55:04.784708 1558 catalog_manager.cc:1127] Tablet: 33c9287806eb49b993d596ec809738c1 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "2a0fe46c06ed4be8bcb0c977eb160888" role: LEADER last_known_addr { host: "127.0.0.1" port: 40525 } } I1102 12:55:04.786715 1558 catalog_manager.cc:1127] Tablet: 11710bcc05da4accb284fefc5032d8f6 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "2a0fe46c06ed4be8bcb0c977eb160888" role: LEADER last_known_addr { host: "127.0.0.1" port: 40525 } } I1102 12:55:04.796144 1563 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:55279: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:04.802805 1609 tablet_service.cc:264] Processing CreateTablet for tablet ca6995f8752644a68fee54c3c2facf05 (table=client-testtb2 [id=7550af3927c34837a7e9b7717c79135d]), range=["", ""] I1102 12:55:04.804191 1609 ts_tablet_manager.cc:460] Registered tablet ca6995f8752644a68fee54c3c2facf05 I1102 12:55:04.804321 1638 ts_tablet_manager.cc:325] Bootstrapping tablet: ca6995f8752644a68fee54c3c2facf05 I1102 12:55:04.805016 1638 tablet_bootstrap.cc:234] Tablet ca6995f8752644a68fee54c3c2facf05: Bootstrap starting. I1102 12:55:04.810005 1638 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: ca6995f8752644a68fee54c3c2facf05 creating new one. I1102 12:55:04.810947 1638 tablet_bootstrap.cc:234] Tablet ca6995f8752644a68fee54c3c2facf05: No bootstrap required, opened a new log I1102 12:55:04.811120 1638 ts_tablet_manager.cc:330] Time spent tablet ca6995f8752644a68fee54c3c2facf05 bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:55:04.812055 1638 ts_tablet_manager.cc:350] Time spent starting tablet ca6995f8752644a68fee54c3c2facf05: real 0.001s user 0.000s sys 0.000s I1102 12:55:04.812296 1649 tablet_peer.cc:213] Configuration changed for tablet: ca6995f8752644a68fee54c3c2facf05 in peer: 2a0fe46c06ed4be8bcb0c977eb160888 Changed from: seqno: -1 local: true peers { permanent_uuid: "2a0fe46c06ed4be8bcb0c977eb160888" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 40525 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "2a0fe46c06ed4be8bcb0c977eb160888" role: LEADER last_known_addr { host: "127.0.0.1" port: 40525 } } I1102 12:55:04.821159 1564 catalog_manager.cc:1127] Tablet: ca6995f8752644a68fee54c3c2facf05 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "2a0fe46c06ed4be8bcb0c977eb160888" role: LEADER last_known_addr { host: "127.0.0.1" port: 40525 } } W1102 12:55:04.836956 31819 client.cc:427] Closing Session with pending operations. I1102 12:55:04.840528 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:04.852872 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:04.852929 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:04.853034 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: ca6995f8752644a68fee54c3c2facf05 I1102 12:55:04.853061 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(ca6995f8752644a68fee54c3c2facf05) I1102 12:55:04.853080 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(ca6995f8752644a68fee54c3c2facf05) I1102 12:55:04.853092 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(ca6995f8752644a68fee54c3c2facf05) I1102 12:55:04.853552 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 33c9287806eb49b993d596ec809738c1 I1102 12:55:04.853582 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(33c9287806eb49b993d596ec809738c1) I1102 12:55:04.853595 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(33c9287806eb49b993d596ec809738c1) I1102 12:55:04.853610 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(33c9287806eb49b993d596ec809738c1) I1102 12:55:04.854182 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11710bcc05da4accb284fefc5032d8f6 I1102 12:55:04.854204 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(11710bcc05da4accb284fefc5032d8f6) I1102 12:55:04.854223 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(11710bcc05da4accb284fefc5032d8f6) I1102 12:55:04.854233 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(11710bcc05da4accb284fefc5032d8f6) I1102 12:55:04.857868 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:04.857892 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:04.857919 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:04.858793 31819 master.cc:124] Master@127.0.0.1:58002 shutting down... I1102 12:55:04.870235 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:04.870656 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:04.871047 31819 master.cc:127] Master@127.0.0.1:58002 shutdown complete. [ OK ] ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered (291 ms) [ RUN ] ClientTest.TestMutationsWork I1102 12:55:05.030622 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestMutationsWork.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "a4bd2f19f21f4432818e3612983c753d" format_stamp: "Formatted at 2014-11-02 20:55:04 on qiwi" I1102 12:55:05.030725 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestMutationsWork.1414961696528764-31819/minicluster-data/master-0-root uuid: "a4bd2f19f21f4432818e3612983c753d" format_stamp: "Formatted at 2014-11-02 20:55:04 on qiwi" I1102 12:55:05.031575 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 495778 I1102 12:55:05.034303 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:05.034318 31819 webserver.cc:117] Document root disabled I1102 12:55:05.034517 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:37511 I1102 12:55:05.035768 1678 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:05.037787 1678 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:05.038638 1678 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:05.040144 1680 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: a4bd2f19f21f4432818e3612983c753d Changed from: seqno: 0 local: true peers { permanent_uuid: "a4bd2f19f21f4432818e3612983c753d" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "a4bd2f19f21f4432818e3612983c753d" role: LEADER } I1102 12:55:05.040202 1680 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "a4bd2f19f21f4432818e3612983c753d" role: LEADER } I1102 12:55:05.040566 1678 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:05.042567 1678 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:05.043406 1678 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:05.044420 1678 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:05.044821 1685 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: a4bd2f19f21f4432818e3612983c753d Changed from: seqno: 0 local: true peers { permanent_uuid: "a4bd2f19f21f4432818e3612983c753d" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "a4bd2f19f21f4432818e3612983c753d" role: LEADER } I1102 12:55:05.044880 1678 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:05.044883 1685 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "a4bd2f19f21f4432818e3612983c753d" role: LEADER } I1102 12:55:05.057035 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestMutationsWork.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "afd53eaf046346b38b78e7947f536e8d" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.057135 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestMutationsWork.1414961696528764-31819/minicluster-data/ts-0-root uuid: "afd53eaf046346b38b78e7947f536e8d" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.057993 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 495778 I1102 12:55:05.062295 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:05.062314 31819 webserver.cc:117] Document root disabled I1102 12:55:05.062540 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:40989 I1102 12:55:05.062671 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:05.064144 1730 heartbeater.cc:227] Registering TS with master... I1102 12:55:05.064195 1730 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:05.064514 1657 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "afd53eaf046346b38b78e7947f536e8d" instance_seqno: 1414961705062025 } with Master I1102 12:55:05.065016 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00216081s I1102 12:55:05.067853 1659 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:45306: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:05.073366 1707 tablet_service.cc:264] Processing CreateTablet for tablet ad0777bbc8564365b9462cae7eb31ac8 (table=client-testtb [id=5af275b256e24f2f98ebb5292e9a9292]), range=["\x00\x00\x00\t", ""] I1102 12:55:05.073422 1708 tablet_service.cc:264] Processing CreateTablet for tablet 5b0882cae45b45e9b5e7e77cbfb34041 (table=client-testtb [id=5af275b256e24f2f98ebb5292e9a9292]), range=["", "\x00\x00\x00\t"] I1102 12:55:05.074275 1707 ts_tablet_manager.cc:460] Registered tablet ad0777bbc8564365b9462cae7eb31ac8 I1102 12:55:05.074328 1708 ts_tablet_manager.cc:460] Registered tablet 5b0882cae45b45e9b5e7e77cbfb34041 I1102 12:55:05.074739 1739 ts_tablet_manager.cc:325] Bootstrapping tablet: ad0777bbc8564365b9462cae7eb31ac8 I1102 12:55:05.074764 1740 ts_tablet_manager.cc:325] Bootstrapping tablet: 5b0882cae45b45e9b5e7e77cbfb34041 I1102 12:55:05.075155 1739 tablet_bootstrap.cc:234] Tablet ad0777bbc8564365b9462cae7eb31ac8: Bootstrap starting. I1102 12:55:05.075223 1740 tablet_bootstrap.cc:234] Tablet 5b0882cae45b45e9b5e7e77cbfb34041: Bootstrap starting. I1102 12:55:05.077366 1739 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: ad0777bbc8564365b9462cae7eb31ac8 creating new one. I1102 12:55:05.077404 1740 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 5b0882cae45b45e9b5e7e77cbfb34041 creating new one. I1102 12:55:05.078554 1740 tablet_bootstrap.cc:234] Tablet 5b0882cae45b45e9b5e7e77cbfb34041: No bootstrap required, opened a new log I1102 12:55:05.078608 1739 tablet_bootstrap.cc:234] Tablet ad0777bbc8564365b9462cae7eb31ac8: No bootstrap required, opened a new log I1102 12:55:05.078775 1740 ts_tablet_manager.cc:330] Time spent tablet 5b0882cae45b45e9b5e7e77cbfb34041 bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:55:05.078799 1739 ts_tablet_manager.cc:330] Time spent tablet ad0777bbc8564365b9462cae7eb31ac8 bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:55:05.079993 1740 ts_tablet_manager.cc:350] Time spent starting tablet 5b0882cae45b45e9b5e7e77cbfb34041: real 0.001s user 0.001s sys 0.000s I1102 12:55:05.080054 1739 ts_tablet_manager.cc:350] Time spent starting tablet ad0777bbc8564365b9462cae7eb31ac8: real 0.001s user 0.001s sys 0.000s I1102 12:55:05.080412 1743 tablet_peer.cc:213] Configuration changed for tablet: 5b0882cae45b45e9b5e7e77cbfb34041 in peer: afd53eaf046346b38b78e7947f536e8d Changed from: seqno: -1 local: true peers { permanent_uuid: "afd53eaf046346b38b78e7947f536e8d" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 36513 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "afd53eaf046346b38b78e7947f536e8d" role: LEADER last_known_addr { host: "127.0.0.1" port: 36513 } } I1102 12:55:05.080584 1744 tablet_peer.cc:213] Configuration changed for tablet: ad0777bbc8564365b9462cae7eb31ac8 in peer: afd53eaf046346b38b78e7947f536e8d Changed from: seqno: -1 local: true peers { permanent_uuid: "afd53eaf046346b38b78e7947f536e8d" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 36513 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "afd53eaf046346b38b78e7947f536e8d" role: LEADER last_known_addr { host: "127.0.0.1" port: 36513 } } I1102 12:55:05.085933 1657 catalog_manager.cc:1127] Tablet: ad0777bbc8564365b9462cae7eb31ac8 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "afd53eaf046346b38b78e7947f536e8d" role: LEADER last_known_addr { host: "127.0.0.1" port: 36513 } } I1102 12:55:05.087368 1657 catalog_manager.cc:1127] Tablet: 5b0882cae45b45e9b5e7e77cbfb34041 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "afd53eaf046346b38b78e7947f536e8d" role: LEADER last_known_addr { host: "127.0.0.1" port: 36513 } } I1102 12:55:05.090421 1661 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:45306: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:05.094389 1709 tablet_service.cc:264] Processing CreateTablet for tablet c2f4924877224d31866b75af6a842cd1 (table=client-testtb2 [id=c1fe0b72446041a3bf33803b62d2483f]), range=["", ""] I1102 12:55:05.095315 1709 ts_tablet_manager.cc:460] Registered tablet c2f4924877224d31866b75af6a842cd1 I1102 12:55:05.095393 1740 ts_tablet_manager.cc:325] Bootstrapping tablet: c2f4924877224d31866b75af6a842cd1 I1102 12:55:05.095770 1740 tablet_bootstrap.cc:234] Tablet c2f4924877224d31866b75af6a842cd1: Bootstrap starting. I1102 12:55:05.097821 1740 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: c2f4924877224d31866b75af6a842cd1 creating new one. I1102 12:55:05.098719 1740 tablet_bootstrap.cc:234] Tablet c2f4924877224d31866b75af6a842cd1: No bootstrap required, opened a new log I1102 12:55:05.098912 1740 ts_tablet_manager.cc:330] Time spent tablet c2f4924877224d31866b75af6a842cd1 bootstrap: real 0.003s user 0.002s sys 0.001s I1102 12:55:05.099947 1740 ts_tablet_manager.cc:350] Time spent starting tablet c2f4924877224d31866b75af6a842cd1: real 0.001s user 0.001s sys 0.000s I1102 12:55:05.100216 1750 tablet_peer.cc:213] Configuration changed for tablet: c2f4924877224d31866b75af6a842cd1 in peer: afd53eaf046346b38b78e7947f536e8d Changed from: seqno: -1 local: true peers { permanent_uuid: "afd53eaf046346b38b78e7947f536e8d" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 36513 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "afd53eaf046346b38b78e7947f536e8d" role: LEADER last_known_addr { host: "127.0.0.1" port: 36513 } } I1102 12:55:05.110115 1659 catalog_manager.cc:1127] Tablet: c2f4924877224d31866b75af6a842cd1 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "afd53eaf046346b38b78e7947f536e8d" role: LEADER last_known_addr { host: "127.0.0.1" port: 36513 } } W1102 12:55:05.118831 1735 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:36513, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:05.122462 1735 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:36513, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:05.128622 1735 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:36513, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} I1102 12:55:05.134371 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:05.152278 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:05.152436 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:05.152523 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: ad0777bbc8564365b9462cae7eb31ac8 I1102 12:55:05.152596 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(ad0777bbc8564365b9462cae7eb31ac8) I1102 12:55:05.152735 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(ad0777bbc8564365b9462cae7eb31ac8) I1102 12:55:05.152803 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(ad0777bbc8564365b9462cae7eb31ac8) I1102 12:55:05.154279 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: c2f4924877224d31866b75af6a842cd1 I1102 12:55:05.154351 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(c2f4924877224d31866b75af6a842cd1) I1102 12:55:05.154419 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(c2f4924877224d31866b75af6a842cd1) I1102 12:55:05.154476 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(c2f4924877224d31866b75af6a842cd1) I1102 12:55:05.155591 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 5b0882cae45b45e9b5e7e77cbfb34041 I1102 12:55:05.155685 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(5b0882cae45b45e9b5e7e77cbfb34041) I1102 12:55:05.155728 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(5b0882cae45b45e9b5e7e77cbfb34041) I1102 12:55:05.155776 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(5b0882cae45b45e9b5e7e77cbfb34041) I1102 12:55:05.164744 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:05.164784 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:05.164856 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:05.166463 31819 master.cc:124] Master@127.0.0.1:46258 shutting down... I1102 12:55:05.179996 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:05.180979 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:05.181937 31819 master.cc:127] Master@127.0.0.1:46258 shutdown complete. [ OK ] ClientTest.TestMutationsWork (312 ms) [ RUN ] ClientTest.TestMutateDeletedRow I1102 12:55:05.334444 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestMutateDeletedRow.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "e0f785287df445d387727b10e1f7735e" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.334548 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestMutateDeletedRow.1414961696528764-31819/minicluster-data/master-0-root uuid: "e0f785287df445d387727b10e1f7735e" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.335388 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 495778 I1102 12:55:05.338382 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:05.338405 31819 webserver.cc:117] Document root disabled I1102 12:55:05.338615 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:46178 I1102 12:55:05.339850 1779 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:05.341754 1779 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:05.342694 1779 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:05.344065 1781 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: e0f785287df445d387727b10e1f7735e Changed from: seqno: 0 local: true peers { permanent_uuid: "e0f785287df445d387727b10e1f7735e" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "e0f785287df445d387727b10e1f7735e" role: LEADER } I1102 12:55:05.344130 1781 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "e0f785287df445d387727b10e1f7735e" role: LEADER } I1102 12:55:05.344528 1779 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:05.346376 1779 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:05.347221 1779 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:05.348196 1779 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:05.348583 1785 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: e0f785287df445d387727b10e1f7735e Changed from: seqno: 0 local: true peers { permanent_uuid: "e0f785287df445d387727b10e1f7735e" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "e0f785287df445d387727b10e1f7735e" role: LEADER } I1102 12:55:05.348660 1779 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:05.348657 1785 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "e0f785287df445d387727b10e1f7735e" role: LEADER } I1102 12:55:05.359661 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestMutateDeletedRow.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "ab99869f7d6545efa28651dcda7ad3c7" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.359756 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestMutateDeletedRow.1414961696528764-31819/minicluster-data/ts-0-root uuid: "ab99869f7d6545efa28651dcda7ad3c7" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.360934 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 495778 I1102 12:55:05.365389 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:05.365413 31819 webserver.cc:117] Document root disabled I1102 12:55:05.365623 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:36204 I1102 12:55:05.365758 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:05.367202 1833 heartbeater.cc:227] Registering TS with master... I1102 12:55:05.367262 1833 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:05.367555 1758 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "ab99869f7d6545efa28651dcda7ad3c7" instance_seqno: 1414961705365123 } with Master I1102 12:55:05.368095 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.0021645s I1102 12:55:05.372508 1760 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:35005: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:05.383255 1809 tablet_service.cc:264] Processing CreateTablet for tablet bc3e0b2d97c24e7a88438f4d165bd822 (table=client-testtb [id=33eb6d7260a04dfe812c6939d88491ec]), range=["\x00\x00\x00\t", ""] I1102 12:55:05.383288 1810 tablet_service.cc:264] Processing CreateTablet for tablet d495b7a52a1a43e1bec268340b92b66d (table=client-testtb [id=33eb6d7260a04dfe812c6939d88491ec]), range=["", "\x00\x00\x00\t"] I1102 12:55:05.384865 1810 ts_tablet_manager.cc:460] Registered tablet d495b7a52a1a43e1bec268340b92b66d I1102 12:55:05.384968 1809 ts_tablet_manager.cc:460] Registered tablet bc3e0b2d97c24e7a88438f4d165bd822 I1102 12:55:05.385318 1842 ts_tablet_manager.cc:325] Bootstrapping tablet: d495b7a52a1a43e1bec268340b92b66d I1102 12:55:05.385670 1843 ts_tablet_manager.cc:325] Bootstrapping tablet: bc3e0b2d97c24e7a88438f4d165bd822 I1102 12:55:05.386327 1842 tablet_bootstrap.cc:234] Tablet d495b7a52a1a43e1bec268340b92b66d: Bootstrap starting. I1102 12:55:05.386530 1843 tablet_bootstrap.cc:234] Tablet bc3e0b2d97c24e7a88438f4d165bd822: Bootstrap starting. I1102 12:55:05.390645 1842 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: d495b7a52a1a43e1bec268340b92b66d creating new one. I1102 12:55:05.390739 1843 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: bc3e0b2d97c24e7a88438f4d165bd822 creating new one. I1102 12:55:05.392767 1843 tablet_bootstrap.cc:234] Tablet bc3e0b2d97c24e7a88438f4d165bd822: No bootstrap required, opened a new log I1102 12:55:05.392781 1842 tablet_bootstrap.cc:234] Tablet d495b7a52a1a43e1bec268340b92b66d: No bootstrap required, opened a new log I1102 12:55:05.393184 1843 ts_tablet_manager.cc:330] Time spent tablet bc3e0b2d97c24e7a88438f4d165bd822 bootstrap: real 0.007s user 0.005s sys 0.002s I1102 12:55:05.393193 1842 ts_tablet_manager.cc:330] Time spent tablet d495b7a52a1a43e1bec268340b92b66d bootstrap: real 0.008s user 0.006s sys 0.001s I1102 12:55:05.395359 1843 ts_tablet_manager.cc:350] Time spent starting tablet bc3e0b2d97c24e7a88438f4d165bd822: real 0.002s user 0.001s sys 0.000s I1102 12:55:05.395421 1842 ts_tablet_manager.cc:350] Time spent starting tablet d495b7a52a1a43e1bec268340b92b66d: real 0.002s user 0.001s sys 0.001s I1102 12:55:05.396297 1848 tablet_peer.cc:213] Configuration changed for tablet: d495b7a52a1a43e1bec268340b92b66d in peer: ab99869f7d6545efa28651dcda7ad3c7 Changed from: seqno: -1 local: true peers { permanent_uuid: "ab99869f7d6545efa28651dcda7ad3c7" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 46444 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "ab99869f7d6545efa28651dcda7ad3c7" role: LEADER last_known_addr { host: "127.0.0.1" port: 46444 } } I1102 12:55:05.396347 1846 tablet_peer.cc:213] Configuration changed for tablet: bc3e0b2d97c24e7a88438f4d165bd822 in peer: ab99869f7d6545efa28651dcda7ad3c7 Changed from: seqno: -1 local: true peers { permanent_uuid: "ab99869f7d6545efa28651dcda7ad3c7" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 46444 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "ab99869f7d6545efa28651dcda7ad3c7" role: LEADER last_known_addr { host: "127.0.0.1" port: 46444 } } I1102 12:55:05.400655 1760 catalog_manager.cc:1127] Tablet: bc3e0b2d97c24e7a88438f4d165bd822 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "ab99869f7d6545efa28651dcda7ad3c7" role: LEADER last_known_addr { host: "127.0.0.1" port: 46444 } } I1102 12:55:05.402209 1760 catalog_manager.cc:1127] Tablet: d495b7a52a1a43e1bec268340b92b66d reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "ab99869f7d6545efa28651dcda7ad3c7" role: LEADER last_known_addr { host: "127.0.0.1" port: 46444 } } I1102 12:55:05.406257 1764 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:35005: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:05.411875 1811 tablet_service.cc:264] Processing CreateTablet for tablet 3abc2f01a8f84843b62aac87729cb9eb (table=client-testtb2 [id=c968ec6de7e7447abb191dbe20a3336c]), range=["", ""] I1102 12:55:05.413450 1811 ts_tablet_manager.cc:460] Registered tablet 3abc2f01a8f84843b62aac87729cb9eb I1102 12:55:05.413615 1843 ts_tablet_manager.cc:325] Bootstrapping tablet: 3abc2f01a8f84843b62aac87729cb9eb I1102 12:55:05.414420 1843 tablet_bootstrap.cc:234] Tablet 3abc2f01a8f84843b62aac87729cb9eb: Bootstrap starting. I1102 12:55:05.418608 1843 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 3abc2f01a8f84843b62aac87729cb9eb creating new one. I1102 12:55:05.420769 1843 tablet_bootstrap.cc:234] Tablet 3abc2f01a8f84843b62aac87729cb9eb: No bootstrap required, opened a new log I1102 12:55:05.421263 1843 ts_tablet_manager.cc:330] Time spent tablet 3abc2f01a8f84843b62aac87729cb9eb bootstrap: real 0.008s user 0.006s sys 0.001s I1102 12:55:05.422596 1843 ts_tablet_manager.cc:350] Time spent starting tablet 3abc2f01a8f84843b62aac87729cb9eb: real 0.001s user 0.001s sys 0.000s I1102 12:55:05.422965 1855 tablet_peer.cc:213] Configuration changed for tablet: 3abc2f01a8f84843b62aac87729cb9eb in peer: ab99869f7d6545efa28651dcda7ad3c7 Changed from: seqno: -1 local: true peers { permanent_uuid: "ab99869f7d6545efa28651dcda7ad3c7" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 46444 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "ab99869f7d6545efa28651dcda7ad3c7" role: LEADER last_known_addr { host: "127.0.0.1" port: 46444 } } I1102 12:55:05.425554 1761 catalog_manager.cc:1127] Tablet: 3abc2f01a8f84843b62aac87729cb9eb reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "ab99869f7d6545efa28651dcda7ad3c7" role: LEADER last_known_addr { host: "127.0.0.1" port: 46444 } } W1102 12:55:05.434545 1837 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:46444, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:05.442457 1837 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:46444, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:05.455603 1837 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:46444, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:05.462564 1837 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:46444, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} I1102 12:55:05.468657 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:05.482331 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:05.482419 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:05.482466 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: bc3e0b2d97c24e7a88438f4d165bd822 I1102 12:55:05.482501 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(bc3e0b2d97c24e7a88438f4d165bd822) I1102 12:55:05.482524 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(bc3e0b2d97c24e7a88438f4d165bd822) I1102 12:55:05.482542 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(bc3e0b2d97c24e7a88438f4d165bd822) I1102 12:55:05.483360 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: d495b7a52a1a43e1bec268340b92b66d I1102 12:55:05.483392 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(d495b7a52a1a43e1bec268340b92b66d) I1102 12:55:05.483408 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(d495b7a52a1a43e1bec268340b92b66d) I1102 12:55:05.483422 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(d495b7a52a1a43e1bec268340b92b66d) I1102 12:55:05.484097 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 3abc2f01a8f84843b62aac87729cb9eb I1102 12:55:05.484125 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(3abc2f01a8f84843b62aac87729cb9eb) I1102 12:55:05.484140 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(3abc2f01a8f84843b62aac87729cb9eb) I1102 12:55:05.484153 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(3abc2f01a8f84843b62aac87729cb9eb) I1102 12:55:05.489663 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:05.489687 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:05.489722 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:05.490685 31819 master.cc:124] Master@127.0.0.1:48219 shutting down... I1102 12:55:05.503360 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:05.504053 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:05.504637 31819 master.cc:127] Master@127.0.0.1:48219 shutdown complete. [ OK ] ClientTest.TestMutateDeletedRow (319 ms) [ RUN ] ClientTest.TestMutateNonexistentRow I1102 12:55:05.654816 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestMutateNonexistentRow.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "ea46cceef1d4410782ec24450a22015e" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.654959 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestMutateNonexistentRow.1414961696528764-31819/minicluster-data/master-0-root uuid: "ea46cceef1d4410782ec24450a22015e" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.655848 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 495778 I1102 12:55:05.658757 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:05.658774 31819 webserver.cc:117] Document root disabled I1102 12:55:05.658995 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:35521 I1102 12:55:05.660287 1892 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:05.662516 1892 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:05.663466 1892 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:05.665091 1895 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: ea46cceef1d4410782ec24450a22015e Changed from: seqno: 0 local: true peers { permanent_uuid: "ea46cceef1d4410782ec24450a22015e" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "ea46cceef1d4410782ec24450a22015e" role: LEADER } I1102 12:55:05.665169 1895 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "ea46cceef1d4410782ec24450a22015e" role: LEADER } I1102 12:55:05.665684 1892 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:05.667956 1892 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:05.668941 1892 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:05.670148 1892 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:05.670634 1899 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: ea46cceef1d4410782ec24450a22015e Changed from: seqno: 0 local: true peers { permanent_uuid: "ea46cceef1d4410782ec24450a22015e" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "ea46cceef1d4410782ec24450a22015e" role: LEADER } I1102 12:55:05.670696 1892 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:05.670699 1899 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "ea46cceef1d4410782ec24450a22015e" role: LEADER } I1102 12:55:05.682016 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestMutateNonexistentRow.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "0a02cbf069e1420b9d3d69c6bb8b133b" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.682144 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestMutateNonexistentRow.1414961696528764-31819/minicluster-data/ts-0-root uuid: "0a02cbf069e1420b9d3d69c6bb8b133b" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.683483 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 495778 I1102 12:55:05.687877 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:05.687896 31819 webserver.cc:117] Document root disabled I1102 12:55:05.688122 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:35562 I1102 12:55:05.688243 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:05.689668 1945 heartbeater.cc:227] Registering TS with master... I1102 12:55:05.689728 1945 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:05.690068 1870 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "0a02cbf069e1420b9d3d69c6bb8b133b" instance_seqno: 1414961705687610 } with Master I1102 12:55:05.690522 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00215717s I1102 12:55:05.693222 1872 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:38031: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:05.699087 1920 tablet_service.cc:264] Processing CreateTablet for tablet 27f7721939fa4aa288ba1123c44cf4d6 (table=client-testtb [id=8cc278e142c24bcea1e09bc6b28be4c6]), range=["", "\x00\x00\x00\t"] I1102 12:55:05.699115 1921 tablet_service.cc:264] Processing CreateTablet for tablet 97398cf4a42846908b4cfd292dde7a97 (table=client-testtb [id=8cc278e142c24bcea1e09bc6b28be4c6]), range=["\x00\x00\x00\t", ""] I1102 12:55:05.699996 1921 ts_tablet_manager.cc:460] Registered tablet 97398cf4a42846908b4cfd292dde7a97 I1102 12:55:05.700062 1920 ts_tablet_manager.cc:460] Registered tablet 27f7721939fa4aa288ba1123c44cf4d6 I1102 12:55:05.700434 1954 ts_tablet_manager.cc:325] Bootstrapping tablet: 97398cf4a42846908b4cfd292dde7a97 I1102 12:55:05.700439 1955 ts_tablet_manager.cc:325] Bootstrapping tablet: 27f7721939fa4aa288ba1123c44cf4d6 I1102 12:55:05.700935 1955 tablet_bootstrap.cc:234] Tablet 27f7721939fa4aa288ba1123c44cf4d6: Bootstrap starting. I1102 12:55:05.700942 1954 tablet_bootstrap.cc:234] Tablet 97398cf4a42846908b4cfd292dde7a97: Bootstrap starting. I1102 12:55:05.703167 1955 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 27f7721939fa4aa288ba1123c44cf4d6 creating new one. I1102 12:55:05.703198 1954 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 97398cf4a42846908b4cfd292dde7a97 creating new one. I1102 12:55:05.704429 1954 tablet_bootstrap.cc:234] Tablet 97398cf4a42846908b4cfd292dde7a97: No bootstrap required, opened a new log I1102 12:55:05.704486 1955 tablet_bootstrap.cc:234] Tablet 27f7721939fa4aa288ba1123c44cf4d6: No bootstrap required, opened a new log I1102 12:55:05.704620 1954 ts_tablet_manager.cc:330] Time spent tablet 97398cf4a42846908b4cfd292dde7a97 bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:55:05.704663 1955 ts_tablet_manager.cc:330] Time spent tablet 27f7721939fa4aa288ba1123c44cf4d6 bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:55:05.705884 1954 ts_tablet_manager.cc:350] Time spent starting tablet 97398cf4a42846908b4cfd292dde7a97: real 0.001s user 0.001s sys 0.000s I1102 12:55:05.705894 1955 ts_tablet_manager.cc:350] Time spent starting tablet 27f7721939fa4aa288ba1123c44cf4d6: real 0.001s user 0.001s sys 0.000s I1102 12:55:05.706387 1958 tablet_peer.cc:213] Configuration changed for tablet: 97398cf4a42846908b4cfd292dde7a97 in peer: 0a02cbf069e1420b9d3d69c6bb8b133b Changed from: seqno: -1 local: true peers { permanent_uuid: "0a02cbf069e1420b9d3d69c6bb8b133b" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 55167 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "0a02cbf069e1420b9d3d69c6bb8b133b" role: LEADER last_known_addr { host: "127.0.0.1" port: 55167 } } I1102 12:55:05.706401 1959 tablet_peer.cc:213] Configuration changed for tablet: 27f7721939fa4aa288ba1123c44cf4d6 in peer: 0a02cbf069e1420b9d3d69c6bb8b133b Changed from: seqno: -1 local: true peers { permanent_uuid: "0a02cbf069e1420b9d3d69c6bb8b133b" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 55167 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "0a02cbf069e1420b9d3d69c6bb8b133b" role: LEADER last_known_addr { host: "127.0.0.1" port: 55167 } } I1102 12:55:05.711992 1870 catalog_manager.cc:1127] Tablet: 27f7721939fa4aa288ba1123c44cf4d6 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "0a02cbf069e1420b9d3d69c6bb8b133b" role: LEADER last_known_addr { host: "127.0.0.1" port: 55167 } } I1102 12:55:05.714552 1870 catalog_manager.cc:1127] Tablet: 97398cf4a42846908b4cfd292dde7a97 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "0a02cbf069e1420b9d3d69c6bb8b133b" role: LEADER last_known_addr { host: "127.0.0.1" port: 55167 } } I1102 12:55:05.724093 1875 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:38031: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:05.731783 1922 tablet_service.cc:264] Processing CreateTablet for tablet e9c9ad04a44143ba8f4badffaccc2bf1 (table=client-testtb2 [id=a8527c0a92cf4b319f73ec4a9b3c6a2f]), range=["", ""] I1102 12:55:05.733348 1922 ts_tablet_manager.cc:460] Registered tablet e9c9ad04a44143ba8f4badffaccc2bf1 I1102 12:55:05.733512 1954 ts_tablet_manager.cc:325] Bootstrapping tablet: e9c9ad04a44143ba8f4badffaccc2bf1 I1102 12:55:05.734277 1954 tablet_bootstrap.cc:234] Tablet e9c9ad04a44143ba8f4badffaccc2bf1: Bootstrap starting. I1102 12:55:05.738245 1954 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: e9c9ad04a44143ba8f4badffaccc2bf1 creating new one. I1102 12:55:05.739825 1954 tablet_bootstrap.cc:234] Tablet e9c9ad04a44143ba8f4badffaccc2bf1: No bootstrap required, opened a new log I1102 12:55:05.740243 1954 ts_tablet_manager.cc:330] Time spent tablet e9c9ad04a44143ba8f4badffaccc2bf1 bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:55:05.741906 1954 ts_tablet_manager.cc:350] Time spent starting tablet e9c9ad04a44143ba8f4badffaccc2bf1: real 0.002s user 0.001s sys 0.000s I1102 12:55:05.742332 1966 tablet_peer.cc:213] Configuration changed for tablet: e9c9ad04a44143ba8f4badffaccc2bf1 in peer: 0a02cbf069e1420b9d3d69c6bb8b133b Changed from: seqno: -1 local: true peers { permanent_uuid: "0a02cbf069e1420b9d3d69c6bb8b133b" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 55167 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "0a02cbf069e1420b9d3d69c6bb8b133b" role: LEADER last_known_addr { host: "127.0.0.1" port: 55167 } } I1102 12:55:05.750761 1876 catalog_manager.cc:1127] Tablet: e9c9ad04a44143ba8f4badffaccc2bf1 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "0a02cbf069e1420b9d3d69c6bb8b133b" role: LEADER last_known_addr { host: "127.0.0.1" port: 55167 } } W1102 12:55:05.760853 1951 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:55167, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:05.769937 1951 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:55167, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} I1102 12:55:05.776808 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:05.791980 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:05.792086 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:05.792273 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 27f7721939fa4aa288ba1123c44cf4d6 I1102 12:55:05.792330 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(27f7721939fa4aa288ba1123c44cf4d6) I1102 12:55:05.792379 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(27f7721939fa4aa288ba1123c44cf4d6) I1102 12:55:05.792414 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(27f7721939fa4aa288ba1123c44cf4d6) I1102 12:55:05.793349 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 97398cf4a42846908b4cfd292dde7a97 I1102 12:55:05.793400 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(97398cf4a42846908b4cfd292dde7a97) I1102 12:55:05.793450 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(97398cf4a42846908b4cfd292dde7a97) I1102 12:55:05.793481 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(97398cf4a42846908b4cfd292dde7a97) I1102 12:55:05.794435 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: e9c9ad04a44143ba8f4badffaccc2bf1 I1102 12:55:05.794482 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(e9c9ad04a44143ba8f4badffaccc2bf1) I1102 12:55:05.794529 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(e9c9ad04a44143ba8f4badffaccc2bf1) I1102 12:55:05.794560 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(e9c9ad04a44143ba8f4badffaccc2bf1) I1102 12:55:05.801395 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:05.801427 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:05.801483 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:05.807328 31819 master.cc:124] Master@127.0.0.1:50260 shutting down... I1102 12:55:05.820736 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:05.821576 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:05.822279 31819 master.cc:127] Master@127.0.0.1:50260 shutdown complete. [ OK ] ClientTest.TestMutateNonexistentRow (319 ms) [ RUN ] ClientTest.TestWriteWithBadColumn I1102 12:55:05.961081 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithBadColumn.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "db1671ee67df453685ceb63298f632b1" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.961233 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithBadColumn.1414961696528764-31819/minicluster-data/master-0-root uuid: "db1671ee67df453685ceb63298f632b1" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.962220 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 495778 I1102 12:55:05.966084 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:05.966104 31819 webserver.cc:117] Document root disabled I1102 12:55:05.966366 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:49410 I1102 12:55:05.967990 1999 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:05.970705 1999 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:05.971720 1999 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:05.973557 2002 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: db1671ee67df453685ceb63298f632b1 Changed from: seqno: 0 local: true peers { permanent_uuid: "db1671ee67df453685ceb63298f632b1" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "db1671ee67df453685ceb63298f632b1" role: LEADER } I1102 12:55:05.973619 2002 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "db1671ee67df453685ceb63298f632b1" role: LEADER } I1102 12:55:05.974002 1999 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:05.976101 1999 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:05.977008 1999 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:05.978085 1999 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:05.978631 2006 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: db1671ee67df453685ceb63298f632b1 Changed from: seqno: 0 local: true peers { permanent_uuid: "db1671ee67df453685ceb63298f632b1" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "db1671ee67df453685ceb63298f632b1" role: LEADER } I1102 12:55:05.978684 1999 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:05.978689 2006 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "db1671ee67df453685ceb63298f632b1" role: LEADER } I1102 12:55:05.992266 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithBadColumn.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "c01c64dacd754be9a3b2ad6c6a0a0937" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.992365 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithBadColumn.1414961696528764-31819/minicluster-data/ts-0-root uuid: "c01c64dacd754be9a3b2ad6c6a0a0937" format_stamp: "Formatted at 2014-11-02 20:55:05 on qiwi" I1102 12:55:05.993562 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 495778 I1102 12:55:05.998486 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:05.998512 31819 webserver.cc:117] Document root disabled I1102 12:55:05.998850 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:54684 I1102 12:55:05.999078 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:06.001291 2051 heartbeater.cc:227] Registering TS with master... I1102 12:55:06.001384 2051 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:06.001929 1974 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "c01c64dacd754be9a3b2ad6c6a0a0937" instance_seqno: 1414961705998058 } with Master I1102 12:55:06.002629 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00328247s I1102 12:55:06.007091 1976 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:56759: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:06.016551 2029 tablet_service.cc:264] Processing CreateTablet for tablet a77ded5aa7f94f0091a0438b3a763a78 (table=client-testtb [id=9ffd27a81baa4adfb121cf64cde0ab51]), range=["", "\x00\x00\x00\t"] I1102 12:55:06.016621 2030 tablet_service.cc:264] Processing CreateTablet for tablet 8ac3744484be4a72aba1b3b3efb0da30 (table=client-testtb [id=9ffd27a81baa4adfb121cf64cde0ab51]), range=["\x00\x00\x00\t", ""] I1102 12:55:06.018504 2030 ts_tablet_manager.cc:460] Registered tablet 8ac3744484be4a72aba1b3b3efb0da30 I1102 12:55:06.018671 2029 ts_tablet_manager.cc:460] Registered tablet a77ded5aa7f94f0091a0438b3a763a78 I1102 12:55:06.019402 2060 ts_tablet_manager.cc:325] Bootstrapping tablet: 8ac3744484be4a72aba1b3b3efb0da30 I1102 12:55:06.019454 2061 ts_tablet_manager.cc:325] Bootstrapping tablet: a77ded5aa7f94f0091a0438b3a763a78 I1102 12:55:06.020411 2060 tablet_bootstrap.cc:234] Tablet 8ac3744484be4a72aba1b3b3efb0da30: Bootstrap starting. I1102 12:55:06.020453 2061 tablet_bootstrap.cc:234] Tablet a77ded5aa7f94f0091a0438b3a763a78: Bootstrap starting. I1102 12:55:06.025459 2060 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 8ac3744484be4a72aba1b3b3efb0da30 creating new one. I1102 12:55:06.025838 2061 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: a77ded5aa7f94f0091a0438b3a763a78 creating new one. I1102 12:55:06.027340 2060 tablet_bootstrap.cc:234] Tablet 8ac3744484be4a72aba1b3b3efb0da30: No bootstrap required, opened a new log I1102 12:55:06.027520 2061 tablet_bootstrap.cc:234] Tablet a77ded5aa7f94f0091a0438b3a763a78: No bootstrap required, opened a new log I1102 12:55:06.027787 2060 ts_tablet_manager.cc:330] Time spent tablet 8ac3744484be4a72aba1b3b3efb0da30 bootstrap: real 0.008s user 0.006s sys 0.001s I1102 12:55:06.027942 2061 ts_tablet_manager.cc:330] Time spent tablet a77ded5aa7f94f0091a0438b3a763a78 bootstrap: real 0.008s user 0.006s sys 0.001s I1102 12:55:06.030042 2061 ts_tablet_manager.cc:350] Time spent starting tablet a77ded5aa7f94f0091a0438b3a763a78: real 0.002s user 0.002s sys 0.001s I1102 12:55:06.030064 2060 ts_tablet_manager.cc:350] Time spent starting tablet 8ac3744484be4a72aba1b3b3efb0da30: real 0.002s user 0.002s sys 0.001s I1102 12:55:06.030819 2065 tablet_peer.cc:213] Configuration changed for tablet: a77ded5aa7f94f0091a0438b3a763a78 in peer: c01c64dacd754be9a3b2ad6c6a0a0937 Changed from: seqno: -1 local: true peers { permanent_uuid: "c01c64dacd754be9a3b2ad6c6a0a0937" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 44297 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "c01c64dacd754be9a3b2ad6c6a0a0937" role: LEADER last_known_addr { host: "127.0.0.1" port: 44297 } } I1102 12:55:06.031016 2064 tablet_peer.cc:213] Configuration changed for tablet: 8ac3744484be4a72aba1b3b3efb0da30 in peer: c01c64dacd754be9a3b2ad6c6a0a0937 Changed from: seqno: -1 local: true peers { permanent_uuid: "c01c64dacd754be9a3b2ad6c6a0a0937" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 44297 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "c01c64dacd754be9a3b2ad6c6a0a0937" role: LEADER last_known_addr { host: "127.0.0.1" port: 44297 } } I1102 12:55:06.035612 1976 catalog_manager.cc:1127] Tablet: a77ded5aa7f94f0091a0438b3a763a78 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "c01c64dacd754be9a3b2ad6c6a0a0937" role: LEADER last_known_addr { host: "127.0.0.1" port: 44297 } } I1102 12:55:06.038521 1976 catalog_manager.cc:1127] Tablet: 8ac3744484be4a72aba1b3b3efb0da30 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "c01c64dacd754be9a3b2ad6c6a0a0937" role: LEADER last_known_addr { host: "127.0.0.1" port: 44297 } } I1102 12:55:06.050233 1984 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:56759: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:06.058100 2031 tablet_service.cc:264] Processing CreateTablet for tablet 6a8f33183ef24f1fb92eb9ca6a28d169 (table=client-testtb2 [id=145a595ac75d49099b8df54c4eb90d25]), range=["", ""] I1102 12:55:06.060212 2031 ts_tablet_manager.cc:460] Registered tablet 6a8f33183ef24f1fb92eb9ca6a28d169 I1102 12:55:06.060396 2061 ts_tablet_manager.cc:325] Bootstrapping tablet: 6a8f33183ef24f1fb92eb9ca6a28d169 I1102 12:55:06.061431 2061 tablet_bootstrap.cc:234] Tablet 6a8f33183ef24f1fb92eb9ca6a28d169: Bootstrap starting. I1102 12:55:06.066653 2061 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 6a8f33183ef24f1fb92eb9ca6a28d169 creating new one. I1102 12:55:06.068784 2061 tablet_bootstrap.cc:234] Tablet 6a8f33183ef24f1fb92eb9ca6a28d169: No bootstrap required, opened a new log I1102 12:55:06.069310 2061 ts_tablet_manager.cc:330] Time spent tablet 6a8f33183ef24f1fb92eb9ca6a28d169 bootstrap: real 0.009s user 0.007s sys 0.001s I1102 12:55:06.071542 2061 ts_tablet_manager.cc:350] Time spent starting tablet 6a8f33183ef24f1fb92eb9ca6a28d169: real 0.002s user 0.001s sys 0.000s I1102 12:55:06.072280 2071 tablet_peer.cc:213] Configuration changed for tablet: 6a8f33183ef24f1fb92eb9ca6a28d169 in peer: c01c64dacd754be9a3b2ad6c6a0a0937 Changed from: seqno: -1 local: true peers { permanent_uuid: "c01c64dacd754be9a3b2ad6c6a0a0937" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 44297 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "c01c64dacd754be9a3b2ad6c6a0a0937" role: LEADER last_known_addr { host: "127.0.0.1" port: 44297 } } I1102 12:55:06.075233 1982 catalog_manager.cc:1127] Tablet: 6a8f33183ef24f1fb92eb9ca6a28d169 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "c01c64dacd754be9a3b2ad6c6a0a0937" role: LEADER last_known_addr { host: "127.0.0.1" port: 44297 } } I1102 12:55:06.086513 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:06.102494 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:06.102618 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:06.102696 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: a77ded5aa7f94f0091a0438b3a763a78 I1102 12:55:06.102753 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(a77ded5aa7f94f0091a0438b3a763a78) I1102 12:55:06.102792 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(a77ded5aa7f94f0091a0438b3a763a78) I1102 12:55:06.102824 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(a77ded5aa7f94f0091a0438b3a763a78) I1102 12:55:06.103927 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 6a8f33183ef24f1fb92eb9ca6a28d169 I1102 12:55:06.103981 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(6a8f33183ef24f1fb92eb9ca6a28d169) I1102 12:55:06.104007 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(6a8f33183ef24f1fb92eb9ca6a28d169) I1102 12:55:06.104028 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(6a8f33183ef24f1fb92eb9ca6a28d169) I1102 12:55:06.104954 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 8ac3744484be4a72aba1b3b3efb0da30 I1102 12:55:06.105003 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(8ac3744484be4a72aba1b3b3efb0da30) I1102 12:55:06.105031 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(8ac3744484be4a72aba1b3b3efb0da30) I1102 12:55:06.105051 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(8ac3744484be4a72aba1b3b3efb0da30) I1102 12:55:06.114716 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:06.114753 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:06.114807 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:06.116989 31819 master.cc:124] Master@127.0.0.1:55753 shutting down... I1102 12:55:06.130019 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:06.131185 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:06.132165 31819 master.cc:127] Master@127.0.0.1:55753 shutdown complete. [ OK ] ClientTest.TestWriteWithBadColumn (312 ms) [ RUN ] ClientTest.TestWriteWithBadSchema I1102 12:55:06.293190 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithBadSchema.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "f97013b3da0c410c8763299991cd55a7" format_stamp: "Formatted at 2014-11-02 20:55:06 on qiwi" I1102 12:55:06.293300 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithBadSchema.1414961696528764-31819/minicluster-data/master-0-root uuid: "f97013b3da0c410c8763299991cd55a7" format_stamp: "Formatted at 2014-11-02 20:55:06 on qiwi" I1102 12:55:06.294147 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 496278 I1102 12:55:06.296800 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:06.296815 31819 webserver.cc:117] Document root disabled I1102 12:55:06.297006 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:34636 I1102 12:55:06.298267 2099 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:06.300075 2099 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:06.300895 2099 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:06.302227 2101 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: f97013b3da0c410c8763299991cd55a7 Changed from: seqno: 0 local: true peers { permanent_uuid: "f97013b3da0c410c8763299991cd55a7" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "f97013b3da0c410c8763299991cd55a7" role: LEADER } I1102 12:55:06.302281 2101 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "f97013b3da0c410c8763299991cd55a7" role: LEADER } I1102 12:55:06.302677 2099 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:06.304368 2099 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:06.305228 2099 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:06.306196 2099 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:06.306483 2105 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: f97013b3da0c410c8763299991cd55a7 Changed from: seqno: 0 local: true peers { permanent_uuid: "f97013b3da0c410c8763299991cd55a7" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "f97013b3da0c410c8763299991cd55a7" role: LEADER } I1102 12:55:06.306531 2099 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:06.306531 2105 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "f97013b3da0c410c8763299991cd55a7" role: LEADER } I1102 12:55:06.317399 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithBadSchema.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "d06107b6d51f48639fcf6eb0d87d7d63" format_stamp: "Formatted at 2014-11-02 20:55:06 on qiwi" I1102 12:55:06.317492 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestWriteWithBadSchema.1414961696528764-31819/minicluster-data/ts-0-root uuid: "d06107b6d51f48639fcf6eb0d87d7d63" format_stamp: "Formatted at 2014-11-02 20:55:06 on qiwi" I1102 12:55:06.318337 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 496278 I1102 12:55:06.323199 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:06.323225 31819 webserver.cc:117] Document root disabled I1102 12:55:06.323534 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:50361 I1102 12:55:06.323741 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:06.325695 2149 heartbeater.cc:227] Registering TS with master... I1102 12:55:06.325779 2149 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:06.326247 2078 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "d06107b6d51f48639fcf6eb0d87d7d63" instance_seqno: 1414961706322868 } with Master I1102 12:55:06.327208 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00326738s I1102 12:55:06.331127 2080 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:36709: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:06.338722 2127 tablet_service.cc:264] Processing CreateTablet for tablet b392228645ee4a928f30363c7d093e89 (table=client-testtb [id=587d5395dc494caeb1408c124f33065f]), range=["", "\x00\x00\x00\t"] I1102 12:55:06.338788 2128 tablet_service.cc:264] Processing CreateTablet for tablet f9345a55ec7043498304f6985957b3e8 (table=client-testtb [id=587d5395dc494caeb1408c124f33065f]), range=["\x00\x00\x00\t", ""] I1102 12:55:06.339992 2128 ts_tablet_manager.cc:460] Registered tablet f9345a55ec7043498304f6985957b3e8 I1102 12:55:06.340103 2127 ts_tablet_manager.cc:460] Registered tablet b392228645ee4a928f30363c7d093e89 I1102 12:55:06.340525 2158 ts_tablet_manager.cc:325] Bootstrapping tablet: f9345a55ec7043498304f6985957b3e8 I1102 12:55:06.340560 2159 ts_tablet_manager.cc:325] Bootstrapping tablet: b392228645ee4a928f30363c7d093e89 I1102 12:55:06.341048 2158 tablet_bootstrap.cc:234] Tablet f9345a55ec7043498304f6985957b3e8: Bootstrap starting. I1102 12:55:06.341084 2159 tablet_bootstrap.cc:234] Tablet b392228645ee4a928f30363c7d093e89: Bootstrap starting. I1102 12:55:06.343859 2158 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: f9345a55ec7043498304f6985957b3e8 creating new one. I1102 12:55:06.343891 2159 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: b392228645ee4a928f30363c7d093e89 creating new one. I1102 12:55:06.345715 2158 tablet_bootstrap.cc:234] Tablet f9345a55ec7043498304f6985957b3e8: No bootstrap required, opened a new log I1102 12:55:06.345741 2159 tablet_bootstrap.cc:234] Tablet b392228645ee4a928f30363c7d093e89: No bootstrap required, opened a new log I1102 12:55:06.345993 2159 ts_tablet_manager.cc:330] Time spent tablet b392228645ee4a928f30363c7d093e89 bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:06.345995 2158 ts_tablet_manager.cc:330] Time spent tablet f9345a55ec7043498304f6985957b3e8 bootstrap: real 0.005s user 0.004s sys 0.001s I1102 12:55:06.347458 2158 ts_tablet_manager.cc:350] Time spent starting tablet f9345a55ec7043498304f6985957b3e8: real 0.001s user 0.001s sys 0.000s I1102 12:55:06.347553 2159 ts_tablet_manager.cc:350] Time spent starting tablet b392228645ee4a928f30363c7d093e89: real 0.001s user 0.001s sys 0.000s I1102 12:55:06.348223 2163 tablet_peer.cc:213] Configuration changed for tablet: f9345a55ec7043498304f6985957b3e8 in peer: d06107b6d51f48639fcf6eb0d87d7d63 Changed from: seqno: -1 local: true peers { permanent_uuid: "d06107b6d51f48639fcf6eb0d87d7d63" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 47540 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "d06107b6d51f48639fcf6eb0d87d7d63" role: LEADER last_known_addr { host: "127.0.0.1" port: 47540 } } I1102 12:55:06.348255 2162 tablet_peer.cc:213] Configuration changed for tablet: b392228645ee4a928f30363c7d093e89 in peer: d06107b6d51f48639fcf6eb0d87d7d63 Changed from: seqno: -1 local: true peers { permanent_uuid: "d06107b6d51f48639fcf6eb0d87d7d63" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 47540 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "d06107b6d51f48639fcf6eb0d87d7d63" role: LEADER last_known_addr { host: "127.0.0.1" port: 47540 } } I1102 12:55:06.348264 2077 catalog_manager.cc:1127] Tablet: b392228645ee4a928f30363c7d093e89 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "d06107b6d51f48639fcf6eb0d87d7d63" role: LEADER last_known_addr { host: "127.0.0.1" port: 47540 } } I1102 12:55:06.350054 2077 catalog_manager.cc:1127] Tablet: f9345a55ec7043498304f6985957b3e8 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "d06107b6d51f48639fcf6eb0d87d7d63" role: LEADER last_known_addr { host: "127.0.0.1" port: 47540 } } I1102 12:55:06.356703 2082 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:36709: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:06.362054 2129 tablet_service.cc:264] Processing CreateTablet for tablet 5ed775cd7b8c4ff88f1ab48dfb12ad53 (table=client-testtb2 [id=a3d3265cbc0841a887326602517f7dff]), range=["", ""] I1102 12:55:06.363201 2129 ts_tablet_manager.cc:460] Registered tablet 5ed775cd7b8c4ff88f1ab48dfb12ad53 I1102 12:55:06.363317 2158 ts_tablet_manager.cc:325] Bootstrapping tablet: 5ed775cd7b8c4ff88f1ab48dfb12ad53 I1102 12:55:06.363880 2158 tablet_bootstrap.cc:234] Tablet 5ed775cd7b8c4ff88f1ab48dfb12ad53: Bootstrap starting. I1102 12:55:06.367141 2158 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 5ed775cd7b8c4ff88f1ab48dfb12ad53 creating new one. I1102 12:55:06.368360 2158 tablet_bootstrap.cc:234] Tablet 5ed775cd7b8c4ff88f1ab48dfb12ad53: No bootstrap required, opened a new log I1102 12:55:06.368626 2158 ts_tablet_manager.cc:330] Time spent tablet 5ed775cd7b8c4ff88f1ab48dfb12ad53 bootstrap: real 0.005s user 0.004s sys 0.001s I1102 12:55:06.369822 2158 ts_tablet_manager.cc:350] Time spent starting tablet 5ed775cd7b8c4ff88f1ab48dfb12ad53: real 0.001s user 0.001s sys 0.000s I1102 12:55:06.370151 2169 tablet_peer.cc:213] Configuration changed for tablet: 5ed775cd7b8c4ff88f1ab48dfb12ad53 in peer: d06107b6d51f48639fcf6eb0d87d7d63 Changed from: seqno: -1 local: true peers { permanent_uuid: "d06107b6d51f48639fcf6eb0d87d7d63" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 47540 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "d06107b6d51f48639fcf6eb0d87d7d63" role: LEADER last_known_addr { host: "127.0.0.1" port: 47540 } } I1102 12:55:06.376991 2080 catalog_manager.cc:1127] Tablet: 5ed775cd7b8c4ff88f1ab48dfb12ad53 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "d06107b6d51f48639fcf6eb0d87d7d63" role: LEADER last_known_addr { host: "127.0.0.1" port: 47540 } } I1102 12:55:06.387732 2078 catalog_manager.cc:760] Servicing AlterTable request from {real_user=alex, eff_user=} at 127.0.0.1:36709: table { table_name: "client-testtb" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "int_val" } } I1102 12:55:06.391180 2166 tablet.cc:638] Alter schema from Schema [10:key[uint32 NOT NULL], 11:int_val[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL]] version 0 to Schema [10:key[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL]] version 1 I1102 12:55:06.391228 2167 tablet.cc:638] Alter schema from Schema [10:key[uint32 NOT NULL], 11:int_val[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL]] version 0 to Schema [10:key[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL]] version 1 I1102 12:55:06.393878 2167 tablet.cc:583] Flush: entering stage 1 (old memrowset already frozen for inserts) I1102 12:55:06.393878 2166 tablet.cc:583] Flush: entering stage 1 (old memrowset already frozen for inserts) I1102 12:55:06.393915 2167 compaction.cc:549] Selected 1 rowsets to compact: I1102 12:55:06.393934 2166 compaction.cc:549] Selected 1 rowsets to compact: I1102 12:55:06.393936 2167 compaction.cc:552] memrowset(current size on disk: ~0 bytes) I1102 12:55:06.393954 2167 tablet.cc:585] Memstore in-memory size: 1572864 bytes I1102 12:55:06.393954 2166 compaction.cc:552] memrowset(current size on disk: ~0 bytes) I1102 12:55:06.393985 2167 tablet.cc:1018] Flush: entering phase 1 (flushing snapshot) I1102 12:55:06.394007 2166 tablet.cc:585] Memstore in-memory size: 1572864 bytes I1102 12:55:06.394034 2166 tablet.cc:1018] Flush: entering phase 1 (flushing snapshot) I1102 12:55:06.395586 2166 multi_column_writer.cc:71] Opened CFile writer for column key[uint32 NOT NULL] I1102 12:55:06.395587 2167 multi_column_writer.cc:71] Opened CFile writer for column key[uint32 NOT NULL] I1102 12:55:06.395949 2166 multi_column_writer.cc:71] Opened CFile writer for column string_val[string NULLABLE] I1102 12:55:06.395995 2167 multi_column_writer.cc:71] Opened CFile writer for column string_val[string NULLABLE] I1102 12:55:06.396205 2166 multi_column_writer.cc:71] Opened CFile writer for column non_null_with_default[uint32 NOT NULL] I1102 12:55:06.396256 2167 multi_column_writer.cc:71] Opened CFile writer for column non_null_with_default[uint32 NOT NULL] I1102 12:55:06.398627 2166 tablet.cc:1048] Flush resulted in no output rows (all input rows were GCed!) Removing all input rowsets. I1102 12:55:06.398646 2167 tablet.cc:1048] Flush resulted in no output rows (all input rows were GCed!) Removing all input rowsets. I1102 12:55:06.406483 2073 catalog_manager.cc:1786] client-testtb [id=587d5395dc494caeb1408c124f33065f] - Alter table completed version=1 W1102 12:55:06.410397 2153 reactor.cc:199] Client call Write has no timeout set for connection id: {remote=127.0.0.1:47540, service_name=kudu.tserver.TabletServerService, user_credentials={real_user=alex, eff_user=}} W1102 12:55:06.412228 2153 batcher.cc:388] Write(f9345a55ec7043498304f6985957b3e8, 1 ops) failed: Invalid argument: Client provided column int_val[uint32 NOT NULL] not present in tablet I1102 12:55:06.412466 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:06.425060 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:06.425153 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:06.425209 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 5ed775cd7b8c4ff88f1ab48dfb12ad53 I1102 12:55:06.425245 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(5ed775cd7b8c4ff88f1ab48dfb12ad53) I1102 12:55:06.425261 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(5ed775cd7b8c4ff88f1ab48dfb12ad53) I1102 12:55:06.425271 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(5ed775cd7b8c4ff88f1ab48dfb12ad53) I1102 12:55:06.425729 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: b392228645ee4a928f30363c7d093e89 I1102 12:55:06.425747 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(b392228645ee4a928f30363c7d093e89) I1102 12:55:06.425757 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(b392228645ee4a928f30363c7d093e89) I1102 12:55:06.425765 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(b392228645ee4a928f30363c7d093e89) I1102 12:55:06.426276 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: f9345a55ec7043498304f6985957b3e8 I1102 12:55:06.426293 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(f9345a55ec7043498304f6985957b3e8) I1102 12:55:06.426302 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(f9345a55ec7043498304f6985957b3e8) I1102 12:55:06.426311 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(f9345a55ec7043498304f6985957b3e8) I1102 12:55:06.430057 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:06.430078 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:06.430106 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:06.431010 31819 master.cc:124] Master@127.0.0.1:46645 shutting down... I1102 12:55:06.442610 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:06.443099 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:06.443545 31819 master.cc:127] Master@127.0.0.1:46645 shutdown complete. [ OK ] ClientTest.TestWriteWithBadSchema (310 ms) [ RUN ] ClientTest.TestBasicAlterOperations I1102 12:55:06.632459 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestBasicAlterOperations.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "da2dce72c23f4784a84b3625e8c70235" format_stamp: "Formatted at 2014-11-02 20:55:06 on qiwi" I1102 12:55:06.632858 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestBasicAlterOperations.1414961696528764-31819/minicluster-data/master-0-root uuid: "da2dce72c23f4784a84b3625e8c70235" format_stamp: "Formatted at 2014-11-02 20:55:06 on qiwi" I1102 12:55:06.634954 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 496278 I1102 12:55:06.643154 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:06.643196 31819 webserver.cc:117] Document root disabled I1102 12:55:06.643757 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:39022 I1102 12:55:06.646301 2199 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:06.649266 2199 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:06.650441 2199 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:06.652410 2201 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: da2dce72c23f4784a84b3625e8c70235 Changed from: seqno: 0 local: true peers { permanent_uuid: "da2dce72c23f4784a84b3625e8c70235" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "da2dce72c23f4784a84b3625e8c70235" role: LEADER } I1102 12:55:06.652487 2201 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "da2dce72c23f4784a84b3625e8c70235" role: LEADER } I1102 12:55:06.653190 2199 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:06.656158 2199 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:06.657289 2199 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:06.658339 2199 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:06.658740 2205 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: da2dce72c23f4784a84b3625e8c70235 Changed from: seqno: 0 local: true peers { permanent_uuid: "da2dce72c23f4784a84b3625e8c70235" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "da2dce72c23f4784a84b3625e8c70235" role: LEADER } I1102 12:55:06.658793 2199 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:06.658795 2205 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "da2dce72c23f4784a84b3625e8c70235" role: LEADER } I1102 12:55:06.671155 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestBasicAlterOperations.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "03e32129a6e34fcd8c9428a10088d7f1" format_stamp: "Formatted at 2014-11-02 20:55:06 on qiwi" I1102 12:55:06.671398 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestBasicAlterOperations.1414961696528764-31819/minicluster-data/ts-0-root uuid: "03e32129a6e34fcd8c9428a10088d7f1" format_stamp: "Formatted at 2014-11-02 20:55:06 on qiwi" I1102 12:55:06.673079 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 496278 I1102 12:55:06.682986 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:06.683022 31819 webserver.cc:117] Document root disabled I1102 12:55:06.683574 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:51826 I1102 12:55:06.683888 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:06.686276 2249 heartbeater.cc:227] Registering TS with master... I1102 12:55:06.686343 2249 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:06.686696 2178 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "03e32129a6e34fcd8c9428a10088d7f1" instance_seqno: 1414961706682323 } with Master I1102 12:55:06.687579 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00325986s I1102 12:55:06.691181 2180 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:49826: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:06.698210 2226 tablet_service.cc:264] Processing CreateTablet for tablet 759452be395144b4ae585059f5ef2f48 (table=client-testtb [id=2e44589c2c914f48aab65d94a68c14e7]), range=["\x00\x00\x00\t", ""] I1102 12:55:06.698297 2227 tablet_service.cc:264] Processing CreateTablet for tablet 24c0ef0ca8be4bc0bafba4b5fddf6de9 (table=client-testtb [id=2e44589c2c914f48aab65d94a68c14e7]), range=["", "\x00\x00\x00\t"] I1102 12:55:06.699457 2226 ts_tablet_manager.cc:460] Registered tablet 759452be395144b4ae585059f5ef2f48 I1102 12:55:06.699527 2227 ts_tablet_manager.cc:460] Registered tablet 24c0ef0ca8be4bc0bafba4b5fddf6de9 I1102 12:55:06.699795 2258 ts_tablet_manager.cc:325] Bootstrapping tablet: 759452be395144b4ae585059f5ef2f48 I1102 12:55:06.700044 2259 ts_tablet_manager.cc:325] Bootstrapping tablet: 24c0ef0ca8be4bc0bafba4b5fddf6de9 I1102 12:55:06.700435 2258 tablet_bootstrap.cc:234] Tablet 759452be395144b4ae585059f5ef2f48: Bootstrap starting. I1102 12:55:06.700655 2259 tablet_bootstrap.cc:234] Tablet 24c0ef0ca8be4bc0bafba4b5fddf6de9: Bootstrap starting. I1102 12:55:06.703757 2258 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 759452be395144b4ae585059f5ef2f48 creating new one. I1102 12:55:06.703835 2259 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 24c0ef0ca8be4bc0bafba4b5fddf6de9 creating new one. I1102 12:55:06.705327 2259 tablet_bootstrap.cc:234] Tablet 24c0ef0ca8be4bc0bafba4b5fddf6de9: No bootstrap required, opened a new log I1102 12:55:06.705572 2258 tablet_bootstrap.cc:234] Tablet 759452be395144b4ae585059f5ef2f48: No bootstrap required, opened a new log I1102 12:55:06.705627 2259 ts_tablet_manager.cc:330] Time spent tablet 24c0ef0ca8be4bc0bafba4b5fddf6de9 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:06.705896 2258 ts_tablet_manager.cc:330] Time spent tablet 759452be395144b4ae585059f5ef2f48 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:06.707135 2259 ts_tablet_manager.cc:350] Time spent starting tablet 24c0ef0ca8be4bc0bafba4b5fddf6de9: real 0.001s user 0.001s sys 0.000s I1102 12:55:06.707142 2258 ts_tablet_manager.cc:350] Time spent starting tablet 759452be395144b4ae585059f5ef2f48: real 0.001s user 0.001s sys 0.000s I1102 12:55:06.707896 2263 tablet_peer.cc:213] Configuration changed for tablet: 759452be395144b4ae585059f5ef2f48 in peer: 03e32129a6e34fcd8c9428a10088d7f1 Changed from: seqno: -1 local: true peers { permanent_uuid: "03e32129a6e34fcd8c9428a10088d7f1" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 60829 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "03e32129a6e34fcd8c9428a10088d7f1" role: LEADER last_known_addr { host: "127.0.0.1" port: 60829 } } I1102 12:55:06.707906 2262 tablet_peer.cc:213] Configuration changed for tablet: 24c0ef0ca8be4bc0bafba4b5fddf6de9 in peer: 03e32129a6e34fcd8c9428a10088d7f1 Changed from: seqno: -1 local: true peers { permanent_uuid: "03e32129a6e34fcd8c9428a10088d7f1" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 60829 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "03e32129a6e34fcd8c9428a10088d7f1" role: LEADER last_known_addr { host: "127.0.0.1" port: 60829 } } I1102 12:55:06.708585 2177 catalog_manager.cc:1127] Tablet: 759452be395144b4ae585059f5ef2f48 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "03e32129a6e34fcd8c9428a10088d7f1" role: LEADER last_known_addr { host: "127.0.0.1" port: 60829 } } I1102 12:55:06.710681 2177 catalog_manager.cc:1127] Tablet: 24c0ef0ca8be4bc0bafba4b5fddf6de9 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "03e32129a6e34fcd8c9428a10088d7f1" role: LEADER last_known_addr { host: "127.0.0.1" port: 60829 } } I1102 12:55:06.716467 2183 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:49826: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:06.722667 2228 tablet_service.cc:264] Processing CreateTablet for tablet defa2b11c9574694b397f861fe9e12ae (table=client-testtb2 [id=12d00597c3424956b3aa1238a2b0d9d1]), range=["", ""] I1102 12:55:06.724015 2228 ts_tablet_manager.cc:460] Registered tablet defa2b11c9574694b397f861fe9e12ae I1102 12:55:06.724125 2258 ts_tablet_manager.cc:325] Bootstrapping tablet: defa2b11c9574694b397f861fe9e12ae I1102 12:55:06.724711 2258 tablet_bootstrap.cc:234] Tablet defa2b11c9574694b397f861fe9e12ae: Bootstrap starting. I1102 12:55:06.728652 2258 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: defa2b11c9574694b397f861fe9e12ae creating new one. I1102 12:55:06.730183 2258 tablet_bootstrap.cc:234] Tablet defa2b11c9574694b397f861fe9e12ae: No bootstrap required, opened a new log I1102 12:55:06.730556 2258 ts_tablet_manager.cc:330] Time spent tablet defa2b11c9574694b397f861fe9e12ae bootstrap: real 0.006s user 0.005s sys 0.001s I1102 12:55:06.732264 2258 ts_tablet_manager.cc:350] Time spent starting tablet defa2b11c9574694b397f861fe9e12ae: real 0.002s user 0.001s sys 0.000s I1102 12:55:06.732652 2269 tablet_peer.cc:213] Configuration changed for tablet: defa2b11c9574694b397f861fe9e12ae in peer: 03e32129a6e34fcd8c9428a10088d7f1 Changed from: seqno: -1 local: true peers { permanent_uuid: "03e32129a6e34fcd8c9428a10088d7f1" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 60829 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "03e32129a6e34fcd8c9428a10088d7f1" role: LEADER last_known_addr { host: "127.0.0.1" port: 60829 } } I1102 12:55:06.738971 2180 catalog_manager.cc:1127] Tablet: defa2b11c9574694b397f861fe9e12ae reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "03e32129a6e34fcd8c9428a10088d7f1" role: LEADER last_known_addr { host: "127.0.0.1" port: 60829 } } I1102 12:55:06.749033 2186 catalog_manager.cc:760] Servicing AlterTable request from {real_user=alex, eff_user=} at 127.0.0.1:49826: table { table_name: "client-testtb" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "key" } } I1102 12:55:06.750396 2185 catalog_manager.cc:760] Servicing AlterTable request from {real_user=alex, eff_user=} at 127.0.0.1:49826: table { table_name: "client-testtb" } alter_schema_steps { type: RENAME_COLUMN rename_column { old_name: "key" new_name: "key2" } } I1102 12:55:06.751643 2178 catalog_manager.cc:760] Servicing AlterTable request from {real_user=alex, eff_user=} at 127.0.0.1:49826: table { table_name: "client-testtb" } alter_schema_steps { type: RENAME_COLUMN rename_column { old_name: "int_val" new_name: "string_val" } } I1102 12:55:06.753351 2179 catalog_manager.cc:760] Servicing AlterTable request from {real_user=alex, eff_user=} at 127.0.0.1:49826: table { table_name: "client-testtb" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "int_val" } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "new_col" type: UINT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } } I1102 12:55:06.757997 2267 tablet.cc:638] Alter schema from Schema [10:key[uint32 NOT NULL], 11:int_val[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL]] version 0 to Schema [10:key[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL], 14:new_col[uint32 NULLABLE]] version 1 I1102 12:55:06.758165 2266 tablet.cc:638] Alter schema from Schema [10:key[uint32 NOT NULL], 11:int_val[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL]] version 0 to Schema [10:key[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL], 14:new_col[uint32 NULLABLE]] version 1 I1102 12:55:06.761842 2267 tablet.cc:583] Flush: entering stage 1 (old memrowset already frozen for inserts) I1102 12:55:06.761884 2267 compaction.cc:549] Selected 1 rowsets to compact: I1102 12:55:06.761894 2266 tablet.cc:583] Flush: entering stage 1 (old memrowset already frozen for inserts) I1102 12:55:06.761906 2267 compaction.cc:552] memrowset(current size on disk: ~0 bytes) I1102 12:55:06.761927 2266 compaction.cc:549] Selected 1 rowsets to compact: I1102 12:55:06.761929 2267 tablet.cc:585] Memstore in-memory size: 1572864 bytes I1102 12:55:06.761948 2266 compaction.cc:552] memrowset(current size on disk: ~0 bytes) I1102 12:55:06.761952 2267 tablet.cc:1018] Flush: entering phase 1 (flushing snapshot) I1102 12:55:06.761970 2266 tablet.cc:585] Memstore in-memory size: 1572864 bytes I1102 12:55:06.761991 2266 tablet.cc:1018] Flush: entering phase 1 (flushing snapshot) I1102 12:55:06.764281 2266 multi_column_writer.cc:71] Opened CFile writer for column key[uint32 NOT NULL] I1102 12:55:06.764281 2267 multi_column_writer.cc:71] Opened CFile writer for column key[uint32 NOT NULL] I1102 12:55:06.765161 2267 multi_column_writer.cc:71] Opened CFile writer for column string_val[string NULLABLE] I1102 12:55:06.765197 2266 multi_column_writer.cc:71] Opened CFile writer for column string_val[string NULLABLE] I1102 12:55:06.765483 2267 multi_column_writer.cc:71] Opened CFile writer for column non_null_with_default[uint32 NOT NULL] I1102 12:55:06.765522 2266 multi_column_writer.cc:71] Opened CFile writer for column non_null_with_default[uint32 NOT NULL] I1102 12:55:06.765785 2267 multi_column_writer.cc:71] Opened CFile writer for column new_col[uint32 NULLABLE] I1102 12:55:06.765837 2266 multi_column_writer.cc:71] Opened CFile writer for column new_col[uint32 NULLABLE] I1102 12:55:06.768548 2267 tablet.cc:1048] Flush resulted in no output rows (all input rows were GCed!) Removing all input rowsets. I1102 12:55:06.768548 2266 tablet.cc:1048] Flush resulted in no output rows (all input rows were GCed!) Removing all input rowsets. I1102 12:55:06.774202 2173 catalog_manager.cc:1786] client-testtb [id=2e44589c2c914f48aab65d94a68c14e7] - Alter table completed version=1 I1102 12:55:06.777313 2179 catalog_manager.cc:760] Servicing AlterTable request from {real_user=alex, eff_user=} at 127.0.0.1:49826: table { table_name: "client-testtb" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "new_string_val" type: STRING is_nullable: true encoding: GROUP_VARINT compression: DEFAULT_COMPRESSION } } } I1102 12:55:06.778362 2181 catalog_manager.cc:760] Servicing AlterTable request from {real_user=alex, eff_user=} at 127.0.0.1:49826: table { table_name: "client-testtb" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "new_string_val" type: STRING is_nullable: true encoding: PREFIX_ENCODING compression: DEFAULT_COMPRESSION } } } I1102 12:55:06.781324 2267 tablet.cc:638] Alter schema from Schema [10:key[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL], 14:new_col[uint32 NULLABLE]] version 1 to Schema [10:key[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL], 14:new_col[uint32 NULLABLE], 15:new_string_val[string NULLABLE]] version 2 I1102 12:55:06.781378 2266 tablet.cc:638] Alter schema from Schema [10:key[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL], 14:new_col[uint32 NULLABLE]] version 1 to Schema [10:key[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL], 14:new_col[uint32 NULLABLE], 15:new_string_val[string NULLABLE]] version 2 I1102 12:55:06.784356 2266 tablet.cc:583] Flush: entering stage 1 (old memrowset already frozen for inserts) I1102 12:55:06.784356 2267 tablet.cc:583] Flush: entering stage 1 (old memrowset already frozen for inserts) I1102 12:55:06.784391 2266 compaction.cc:549] Selected 1 rowsets to compact: I1102 12:55:06.784395 2267 compaction.cc:549] Selected 1 rowsets to compact: I1102 12:55:06.784404 2266 compaction.cc:552] memrowset(current size on disk: ~0 bytes) I1102 12:55:06.784405 2267 compaction.cc:552] memrowset(current size on disk: ~0 bytes) I1102 12:55:06.784420 2267 tablet.cc:585] Memstore in-memory size: 1572864 bytes I1102 12:55:06.784418 2266 tablet.cc:585] Memstore in-memory size: 1572864 bytes I1102 12:55:06.784436 2267 tablet.cc:1018] Flush: entering phase 1 (flushing snapshot) I1102 12:55:06.784447 2266 tablet.cc:1018] Flush: entering phase 1 (flushing snapshot) I1102 12:55:06.785435 2266 multi_column_writer.cc:71] Opened CFile writer for column key[uint32 NOT NULL] I1102 12:55:06.785435 2267 multi_column_writer.cc:71] Opened CFile writer for column key[uint32 NOT NULL] I1102 12:55:06.785764 2267 multi_column_writer.cc:71] Opened CFile writer for column string_val[string NULLABLE] I1102 12:55:06.785764 2266 multi_column_writer.cc:71] Opened CFile writer for column string_val[string NULLABLE] I1102 12:55:06.786010 2267 multi_column_writer.cc:71] Opened CFile writer for column non_null_with_default[uint32 NOT NULL] I1102 12:55:06.786046 2266 multi_column_writer.cc:71] Opened CFile writer for column non_null_with_default[uint32 NOT NULL] I1102 12:55:06.786278 2267 multi_column_writer.cc:71] Opened CFile writer for column new_col[uint32 NULLABLE] I1102 12:55:06.786312 2266 multi_column_writer.cc:71] Opened CFile writer for column new_col[uint32 NULLABLE] I1102 12:55:06.786545 2267 multi_column_writer.cc:71] Opened CFile writer for column new_string_val[string NULLABLE] I1102 12:55:06.786588 2266 multi_column_writer.cc:71] Opened CFile writer for column new_string_val[string NULLABLE] I1102 12:55:06.788957 2267 tablet.cc:1048] Flush resulted in no output rows (all input rows were GCed!) Removing all input rowsets. I1102 12:55:06.788957 2266 tablet.cc:1048] Flush resulted in no output rows (all input rows were GCed!) Removing all input rowsets. I1102 12:55:06.793722 2173 catalog_manager.cc:1786] client-testtb [id=2e44589c2c914f48aab65d94a68c14e7] - Alter table completed version=2 I1102 12:55:06.794507 2177 catalog_manager.cc:760] Servicing AlterTable request from {real_user=alex, eff_user=} at 127.0.0.1:49826: table { table_name: "client-testtb" } new_table_name: "RenamedTable" I1102 12:55:06.796641 2266 tablet.cc:638] Alter schema from Schema [10:key[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL], 14:new_col[uint32 NULLABLE], 15:new_string_val[string NULLABLE]] version 2 to Schema [10:key[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL], 14:new_col[uint32 NULLABLE], 15:new_string_val[string NULLABLE]] version 3 I1102 12:55:06.796761 2267 tablet.cc:638] Alter schema from Schema [10:key[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL], 14:new_col[uint32 NULLABLE], 15:new_string_val[string NULLABLE]] version 2 to Schema [10:key[uint32 NOT NULL], 12:string_val[string NULLABLE], 13:non_null_with_default[uint32 NOT NULL], 14:new_col[uint32 NULLABLE], 15:new_string_val[string NULLABLE]] version 3 I1102 12:55:06.799345 2173 catalog_manager.cc:1786] RenamedTable [id=2e44589c2c914f48aab65d94a68c14e7] - Alter table completed version=3 I1102 12:55:06.800374 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:06.813138 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:06.813208 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:06.813251 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 759452be395144b4ae585059f5ef2f48 I1102 12:55:06.813276 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(759452be395144b4ae585059f5ef2f48) I1102 12:55:06.813289 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(759452be395144b4ae585059f5ef2f48) I1102 12:55:06.813302 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(759452be395144b4ae585059f5ef2f48) I1102 12:55:06.813828 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: defa2b11c9574694b397f861fe9e12ae I1102 12:55:06.813848 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(defa2b11c9574694b397f861fe9e12ae) I1102 12:55:06.813858 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(defa2b11c9574694b397f861fe9e12ae) I1102 12:55:06.813865 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(defa2b11c9574694b397f861fe9e12ae) I1102 12:55:06.814203 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 24c0ef0ca8be4bc0bafba4b5fddf6de9 I1102 12:55:06.814220 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(24c0ef0ca8be4bc0bafba4b5fddf6de9) I1102 12:55:06.814230 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(24c0ef0ca8be4bc0bafba4b5fddf6de9) I1102 12:55:06.814237 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(24c0ef0ca8be4bc0bafba4b5fddf6de9) I1102 12:55:06.817756 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:06.817778 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:06.817806 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:06.818635 31819 master.cc:124] Master@127.0.0.1:39289 shutting down... I1102 12:55:06.830137 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:06.830653 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:06.831039 31819 master.cc:127] Master@127.0.0.1:39289 shutdown complete. [ OK ] ClientTest.TestBasicAlterOperations (388 ms) [ RUN ] ClientTest.TestDeleteTable I1102 12:55:07.031414 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestDeleteTable.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "f97c60a770804c7cb8960a63191f5ede" format_stamp: "Formatted at 2014-11-02 20:55:06 on qiwi" I1102 12:55:07.031575 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestDeleteTable.1414961696528764-31819/minicluster-data/master-0-root uuid: "f97c60a770804c7cb8960a63191f5ede" format_stamp: "Formatted at 2014-11-02 20:55:06 on qiwi" I1102 12:55:07.032559 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 496778 I1102 12:55:07.036339 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:07.036360 31819 webserver.cc:117] Document root disabled I1102 12:55:07.036602 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:44706 I1102 12:55:07.038213 2300 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:07.040766 2300 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:07.041740 2300 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:07.043364 2302 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: f97c60a770804c7cb8960a63191f5ede Changed from: seqno: 0 local: true peers { permanent_uuid: "f97c60a770804c7cb8960a63191f5ede" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "f97c60a770804c7cb8960a63191f5ede" role: LEADER } I1102 12:55:07.043424 2302 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "f97c60a770804c7cb8960a63191f5ede" role: LEADER } I1102 12:55:07.043853 2300 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:07.046118 2300 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:07.047082 2300 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:07.048228 2300 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:07.048670 2306 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: f97c60a770804c7cb8960a63191f5ede Changed from: seqno: 0 local: true peers { permanent_uuid: "f97c60a770804c7cb8960a63191f5ede" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "f97c60a770804c7cb8960a63191f5ede" role: LEADER } I1102 12:55:07.048724 2300 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:07.048730 2306 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "f97c60a770804c7cb8960a63191f5ede" role: LEADER } I1102 12:55:07.061794 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestDeleteTable.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "3db90716f09f446e90f0d6030dd681fe" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:07.061880 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestDeleteTable.1414961696528764-31819/minicluster-data/ts-0-root uuid: "3db90716f09f446e90f0d6030dd681fe" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:07.062635 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 496778 I1102 12:55:07.066375 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:07.066391 31819 webserver.cc:117] Document root disabled I1102 12:55:07.066611 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:60015 I1102 12:55:07.066736 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:07.068171 2354 heartbeater.cc:227] Registering TS with master... I1102 12:55:07.068222 2354 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:07.068498 2277 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "3db90716f09f446e90f0d6030dd681fe" instance_seqno: 1414961707066118 } with Master I1102 12:55:07.069010 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00215476s I1102 12:55:07.072510 2279 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:54262: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:07.079390 2329 tablet_service.cc:264] Processing CreateTablet for tablet 357655ed138344bc97817ef28216e3f4 (table=client-testtb [id=2acb097721dd4f38ae704e637f735a28]), range=["\x00\x00\x00\t", ""] I1102 12:55:07.079426 2330 tablet_service.cc:264] Processing CreateTablet for tablet 6942f47466cb461f9e6a752baa030a9e (table=client-testtb [id=2acb097721dd4f38ae704e637f735a28]), range=["", "\x00\x00\x00\t"] I1102 12:55:07.080507 2329 ts_tablet_manager.cc:460] Registered tablet 357655ed138344bc97817ef28216e3f4 I1102 12:55:07.080539 2330 ts_tablet_manager.cc:460] Registered tablet 6942f47466cb461f9e6a752baa030a9e I1102 12:55:07.080792 2363 ts_tablet_manager.cc:325] Bootstrapping tablet: 357655ed138344bc97817ef28216e3f4 I1102 12:55:07.081017 2364 ts_tablet_manager.cc:325] Bootstrapping tablet: 6942f47466cb461f9e6a752baa030a9e I1102 12:55:07.081370 2363 tablet_bootstrap.cc:234] Tablet 357655ed138344bc97817ef28216e3f4: Bootstrap starting. I1102 12:55:07.081531 2364 tablet_bootstrap.cc:234] Tablet 6942f47466cb461f9e6a752baa030a9e: Bootstrap starting. I1102 12:55:07.084096 2363 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 357655ed138344bc97817ef28216e3f4 creating new one. I1102 12:55:07.084167 2364 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 6942f47466cb461f9e6a752baa030a9e creating new one. I1102 12:55:07.085955 2363 tablet_bootstrap.cc:234] Tablet 357655ed138344bc97817ef28216e3f4: No bootstrap required, opened a new log I1102 12:55:07.086012 2364 tablet_bootstrap.cc:234] Tablet 6942f47466cb461f9e6a752baa030a9e: No bootstrap required, opened a new log I1102 12:55:07.086297 2363 ts_tablet_manager.cc:330] Time spent tablet 357655ed138344bc97817ef28216e3f4 bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:07.086341 2364 ts_tablet_manager.cc:330] Time spent tablet 6942f47466cb461f9e6a752baa030a9e bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:07.088193 2364 ts_tablet_manager.cc:350] Time spent starting tablet 6942f47466cb461f9e6a752baa030a9e: real 0.002s user 0.001s sys 0.001s I1102 12:55:07.088260 2363 ts_tablet_manager.cc:350] Time spent starting tablet 357655ed138344bc97817ef28216e3f4: real 0.002s user 0.001s sys 0.001s I1102 12:55:07.088997 2368 tablet_peer.cc:213] Configuration changed for tablet: 357655ed138344bc97817ef28216e3f4 in peer: 3db90716f09f446e90f0d6030dd681fe Changed from: seqno: -1 local: true peers { permanent_uuid: "3db90716f09f446e90f0d6030dd681fe" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 56013 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "3db90716f09f446e90f0d6030dd681fe" role: LEADER last_known_addr { host: "127.0.0.1" port: 56013 } } I1102 12:55:07.089006 2367 tablet_peer.cc:213] Configuration changed for tablet: 6942f47466cb461f9e6a752baa030a9e in peer: 3db90716f09f446e90f0d6030dd681fe Changed from: seqno: -1 local: true peers { permanent_uuid: "3db90716f09f446e90f0d6030dd681fe" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 56013 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "3db90716f09f446e90f0d6030dd681fe" role: LEADER last_known_addr { host: "127.0.0.1" port: 56013 } } I1102 12:55:07.090047 2276 catalog_manager.cc:1127] Tablet: 357655ed138344bc97817ef28216e3f4 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "3db90716f09f446e90f0d6030dd681fe" role: LEADER last_known_addr { host: "127.0.0.1" port: 56013 } } I1102 12:55:07.091786 2276 catalog_manager.cc:1127] Tablet: 6942f47466cb461f9e6a752baa030a9e reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "3db90716f09f446e90f0d6030dd681fe" role: LEADER last_known_addr { host: "127.0.0.1" port: 56013 } } I1102 12:55:07.096834 2281 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:54262: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:07.102747 2331 tablet_service.cc:264] Processing CreateTablet for tablet 0012db673db84162b7cbadc8d95cff48 (table=client-testtb2 [id=a1238334ebf4480e9685b361f36c003e]), range=["", ""] I1102 12:55:07.104625 2331 ts_tablet_manager.cc:460] Registered tablet 0012db673db84162b7cbadc8d95cff48 I1102 12:55:07.104794 2364 ts_tablet_manager.cc:325] Bootstrapping tablet: 0012db673db84162b7cbadc8d95cff48 I1102 12:55:07.105618 2364 tablet_bootstrap.cc:234] Tablet 0012db673db84162b7cbadc8d95cff48: Bootstrap starting. I1102 12:55:07.109917 2364 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 0012db673db84162b7cbadc8d95cff48 creating new one. I1102 12:55:07.111726 2364 tablet_bootstrap.cc:234] Tablet 0012db673db84162b7cbadc8d95cff48: No bootstrap required, opened a new log I1102 12:55:07.112166 2364 ts_tablet_manager.cc:330] Time spent tablet 0012db673db84162b7cbadc8d95cff48 bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:55:07.114068 2364 ts_tablet_manager.cc:350] Time spent starting tablet 0012db673db84162b7cbadc8d95cff48: real 0.002s user 0.001s sys 0.000s I1102 12:55:07.114636 2374 tablet_peer.cc:213] Configuration changed for tablet: 0012db673db84162b7cbadc8d95cff48 in peer: 3db90716f09f446e90f0d6030dd681fe Changed from: seqno: -1 local: true peers { permanent_uuid: "3db90716f09f446e90f0d6030dd681fe" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 56013 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "3db90716f09f446e90f0d6030dd681fe" role: LEADER last_known_addr { host: "127.0.0.1" port: 56013 } } I1102 12:55:07.115809 2278 catalog_manager.cc:1127] Tablet: 0012db673db84162b7cbadc8d95cff48 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "3db90716f09f446e90f0d6030dd681fe" role: LEADER last_known_addr { host: "127.0.0.1" port: 56013 } } I1102 12:55:07.123427 2283 catalog_manager.cc:624] Servicing DeleteTable request from {real_user=alex, eff_user=} at 127.0.0.1:54262: table { table_name: "client-testtb" } I1102 12:55:07.125471 2283 catalog_manager.cc:678] Successfully deleted table client-testtb [id=2acb097721dd4f38ae704e637f735a28] per request from {real_user=alex, eff_user=} at 127.0.0.1:54262 I1102 12:55:07.125797 2332 tablet_service.cc:305] Processing DeleteTablet for tablet 6942f47466cb461f9e6a752baa030a9e (Table being deleted) from {real_user=alex, eff_user=} at 127.0.0.1:35281 I1102 12:55:07.125874 2332 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 6942f47466cb461f9e6a752baa030a9e I1102 12:55:07.125883 2333 tablet_service.cc:305] Processing DeleteTablet for tablet 357655ed138344bc97817ef28216e3f4 (Table being deleted) from {real_user=alex, eff_user=} at 127.0.0.1:35281 I1102 12:55:07.125913 2332 maintenance_manager.cc:177] Unregistered op FlushMRSOp(6942f47466cb461f9e6a752baa030a9e) I1102 12:55:07.125931 2332 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(6942f47466cb461f9e6a752baa030a9e) I1102 12:55:07.125942 2332 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(6942f47466cb461f9e6a752baa030a9e) I1102 12:55:07.125952 2333 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 357655ed138344bc97817ef28216e3f4 I1102 12:55:07.125983 2333 maintenance_manager.cc:177] Unregistered op FlushMRSOp(357655ed138344bc97817ef28216e3f4) I1102 12:55:07.126001 2333 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(357655ed138344bc97817ef28216e3f4) I1102 12:55:07.126013 2333 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(357655ed138344bc97817ef28216e3f4) I1102 12:55:07.127218 2273 catalog_manager.cc:1493] Tablet 6942f47466cb461f9e6a752baa030a9e (table client-testtb [id=2acb097721dd4f38ae704e637f735a28]) successfully deleted I1102 12:55:07.127328 2273 catalog_manager.cc:1493] Tablet 357655ed138344bc97817ef28216e3f4 (table client-testtb [id=2acb097721dd4f38ae704e637f735a28]) successfully deleted I1102 12:55:07.132153 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:07.146618 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:07.146827 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:07.146957 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 0012db673db84162b7cbadc8d95cff48 I1102 12:55:07.147048 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(0012db673db84162b7cbadc8d95cff48) I1102 12:55:07.147105 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(0012db673db84162b7cbadc8d95cff48) I1102 12:55:07.147143 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(0012db673db84162b7cbadc8d95cff48) I1102 12:55:07.150946 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:07.150972 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:07.151007 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:07.152261 31819 master.cc:124] Master@127.0.0.1:57201 shutting down... I1102 12:55:07.165444 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:07.166610 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:07.167606 31819 master.cc:127] Master@127.0.0.1:57201 shutdown complete. [ OK ] ClientTest.TestDeleteTable (336 ms) [ RUN ] ClientTest.TestGetTableSchema I1102 12:55:07.318616 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestGetTableSchema.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "a22ef8bcd9a74af08eda44fb51d413ea" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:07.318866 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestGetTableSchema.1414961696528764-31819/minicluster-data/master-0-root uuid: "a22ef8bcd9a74af08eda44fb51d413ea" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:07.320579 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 496778 I1102 12:55:07.327395 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:07.327432 31819 webserver.cc:117] Document root disabled I1102 12:55:07.327898 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:43820 I1102 12:55:07.330873 2402 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:07.334305 2402 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:07.335204 2402 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:07.336683 2404 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: a22ef8bcd9a74af08eda44fb51d413ea Changed from: seqno: 0 local: true peers { permanent_uuid: "a22ef8bcd9a74af08eda44fb51d413ea" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "a22ef8bcd9a74af08eda44fb51d413ea" role: LEADER } I1102 12:55:07.336748 2404 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "a22ef8bcd9a74af08eda44fb51d413ea" role: LEADER } I1102 12:55:07.337334 2402 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:07.339457 2402 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:07.340328 2402 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:07.341450 2402 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:07.341754 2408 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: a22ef8bcd9a74af08eda44fb51d413ea Changed from: seqno: 0 local: true peers { permanent_uuid: "a22ef8bcd9a74af08eda44fb51d413ea" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "a22ef8bcd9a74af08eda44fb51d413ea" role: LEADER } I1102 12:55:07.341804 2402 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:07.341809 2408 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "a22ef8bcd9a74af08eda44fb51d413ea" role: LEADER } I1102 12:55:07.353775 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestGetTableSchema.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "d2b66682c42c45f08a1945dd59f93aa9" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:07.353870 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestGetTableSchema.1414961696528764-31819/minicluster-data/ts-0-root uuid: "d2b66682c42c45f08a1945dd59f93aa9" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:07.354733 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 496778 I1102 12:55:07.359046 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:07.359063 31819 webserver.cc:117] Document root disabled I1102 12:55:07.359295 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:37680 I1102 12:55:07.359424 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:07.360939 2451 heartbeater.cc:227] Registering TS with master... I1102 12:55:07.360993 2451 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:07.361363 2381 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "d2b66682c42c45f08a1945dd59f93aa9" instance_seqno: 1414961707358776 } with Master I1102 12:55:07.361702 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00215822s I1102 12:55:07.364760 2383 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:37493: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:07.371544 2429 tablet_service.cc:264] Processing CreateTablet for tablet d072dc1f66c34aef8a0af266f8519789 (table=client-testtb [id=54e257b1314d4f37b86f2be3f360af6e]), range=["", "\x00\x00\x00\t"] I1102 12:55:07.371595 2430 tablet_service.cc:264] Processing CreateTablet for tablet 75b1e00265d64325adfa0b451ea11b07 (table=client-testtb [id=54e257b1314d4f37b86f2be3f360af6e]), range=["\x00\x00\x00\t", ""] I1102 12:55:07.372728 2430 ts_tablet_manager.cc:460] Registered tablet 75b1e00265d64325adfa0b451ea11b07 I1102 12:55:07.372791 2429 ts_tablet_manager.cc:460] Registered tablet d072dc1f66c34aef8a0af266f8519789 I1102 12:55:07.373291 2460 ts_tablet_manager.cc:325] Bootstrapping tablet: 75b1e00265d64325adfa0b451ea11b07 I1102 12:55:07.373316 2461 ts_tablet_manager.cc:325] Bootstrapping tablet: d072dc1f66c34aef8a0af266f8519789 I1102 12:55:07.374022 2460 tablet_bootstrap.cc:234] Tablet 75b1e00265d64325adfa0b451ea11b07: Bootstrap starting. I1102 12:55:07.374033 2461 tablet_bootstrap.cc:234] Tablet d072dc1f66c34aef8a0af266f8519789: Bootstrap starting. I1102 12:55:07.377634 2460 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 75b1e00265d64325adfa0b451ea11b07 creating new one. I1102 12:55:07.377677 2461 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: d072dc1f66c34aef8a0af266f8519789 creating new one. I1102 12:55:07.379698 2460 tablet_bootstrap.cc:234] Tablet 75b1e00265d64325adfa0b451ea11b07: No bootstrap required, opened a new log I1102 12:55:07.379787 2461 tablet_bootstrap.cc:234] Tablet d072dc1f66c34aef8a0af266f8519789: No bootstrap required, opened a new log I1102 12:55:07.380097 2460 ts_tablet_manager.cc:330] Time spent tablet 75b1e00265d64325adfa0b451ea11b07 bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:55:07.380136 2461 ts_tablet_manager.cc:330] Time spent tablet d072dc1f66c34aef8a0af266f8519789 bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:55:07.381790 2460 ts_tablet_manager.cc:350] Time spent starting tablet 75b1e00265d64325adfa0b451ea11b07: real 0.002s user 0.001s sys 0.000s I1102 12:55:07.381868 2461 ts_tablet_manager.cc:350] Time spent starting tablet d072dc1f66c34aef8a0af266f8519789: real 0.002s user 0.001s sys 0.001s I1102 12:55:07.382470 2465 tablet_peer.cc:213] Configuration changed for tablet: d072dc1f66c34aef8a0af266f8519789 in peer: d2b66682c42c45f08a1945dd59f93aa9 Changed from: seqno: -1 local: true peers { permanent_uuid: "d2b66682c42c45f08a1945dd59f93aa9" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 38423 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "d2b66682c42c45f08a1945dd59f93aa9" role: LEADER last_known_addr { host: "127.0.0.1" port: 38423 } } I1102 12:55:07.382469 2464 tablet_peer.cc:213] Configuration changed for tablet: 75b1e00265d64325adfa0b451ea11b07 in peer: d2b66682c42c45f08a1945dd59f93aa9 Changed from: seqno: -1 local: true peers { permanent_uuid: "d2b66682c42c45f08a1945dd59f93aa9" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 38423 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "d2b66682c42c45f08a1945dd59f93aa9" role: LEADER last_known_addr { host: "127.0.0.1" port: 38423 } } I1102 12:55:07.383193 2380 catalog_manager.cc:1127] Tablet: d072dc1f66c34aef8a0af266f8519789 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "d2b66682c42c45f08a1945dd59f93aa9" role: LEADER last_known_addr { host: "127.0.0.1" port: 38423 } } I1102 12:55:07.384707 2380 catalog_manager.cc:1127] Tablet: 75b1e00265d64325adfa0b451ea11b07 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "d2b66682c42c45f08a1945dd59f93aa9" role: LEADER last_known_addr { host: "127.0.0.1" port: 38423 } } I1102 12:55:07.389441 2385 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:37493: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:07.394640 2431 tablet_service.cc:264] Processing CreateTablet for tablet e17c446c2dcc46ca8b30744f52939e23 (table=client-testtb2 [id=296a6858c05e405baada0ac6bf0183b6]), range=["", ""] I1102 12:55:07.396726 2431 ts_tablet_manager.cc:460] Registered tablet e17c446c2dcc46ca8b30744f52939e23 I1102 12:55:07.396949 2460 ts_tablet_manager.cc:325] Bootstrapping tablet: e17c446c2dcc46ca8b30744f52939e23 I1102 12:55:07.398064 2460 tablet_bootstrap.cc:234] Tablet e17c446c2dcc46ca8b30744f52939e23: Bootstrap starting. I1102 12:55:07.402592 2460 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: e17c446c2dcc46ca8b30744f52939e23 creating new one. I1102 12:55:07.403961 2460 tablet_bootstrap.cc:234] Tablet e17c446c2dcc46ca8b30744f52939e23: No bootstrap required, opened a new log I1102 12:55:07.404292 2460 ts_tablet_manager.cc:330] Time spent tablet e17c446c2dcc46ca8b30744f52939e23 bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:55:07.405725 2460 ts_tablet_manager.cc:350] Time spent starting tablet e17c446c2dcc46ca8b30744f52939e23: real 0.001s user 0.001s sys 0.000s I1102 12:55:07.406162 2471 tablet_peer.cc:213] Configuration changed for tablet: e17c446c2dcc46ca8b30744f52939e23 in peer: d2b66682c42c45f08a1945dd59f93aa9 Changed from: seqno: -1 local: true peers { permanent_uuid: "d2b66682c42c45f08a1945dd59f93aa9" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 38423 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "d2b66682c42c45f08a1945dd59f93aa9" role: LEADER last_known_addr { host: "127.0.0.1" port: 38423 } } I1102 12:55:07.414114 2384 catalog_manager.cc:1127] Tablet: e17c446c2dcc46ca8b30744f52939e23 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "d2b66682c42c45f08a1945dd59f93aa9" role: LEADER last_known_addr { host: "127.0.0.1" port: 38423 } } I1102 12:55:07.422281 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:07.439688 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:07.439770 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:07.439831 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: d072dc1f66c34aef8a0af266f8519789 I1102 12:55:07.439872 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(d072dc1f66c34aef8a0af266f8519789) I1102 12:55:07.439899 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(d072dc1f66c34aef8a0af266f8519789) I1102 12:55:07.439923 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(d072dc1f66c34aef8a0af266f8519789) I1102 12:55:07.440588 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 75b1e00265d64325adfa0b451ea11b07 I1102 12:55:07.440613 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(75b1e00265d64325adfa0b451ea11b07) I1102 12:55:07.440625 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(75b1e00265d64325adfa0b451ea11b07) I1102 12:55:07.440635 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(75b1e00265d64325adfa0b451ea11b07) I1102 12:55:07.441176 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: e17c446c2dcc46ca8b30744f52939e23 I1102 12:55:07.441198 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(e17c446c2dcc46ca8b30744f52939e23) I1102 12:55:07.441210 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(e17c446c2dcc46ca8b30744f52939e23) I1102 12:55:07.441220 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(e17c446c2dcc46ca8b30744f52939e23) I1102 12:55:07.445402 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:07.445422 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:07.445452 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:07.446635 31819 master.cc:124] Master@127.0.0.1:49241 shutting down... I1102 12:55:07.461835 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:07.462999 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:07.463968 31819 master.cc:127] Master@127.0.0.1:49241 shutdown complete. [ OK ] ClientTest.TestGetTableSchema (296 ms) [ RUN ] ClientTest.TestStaleLocations I1102 12:55:07.615345 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "0717a0d0c74f4a80b28209f66e3ca654" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:07.615770 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root uuid: "0717a0d0c74f4a80b28209f66e3ca654" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:07.618226 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 496778 I1102 12:55:07.627814 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:07.627872 31819 webserver.cc:117] Document root disabled I1102 12:55:07.628619 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:50716 I1102 12:55:07.630763 2500 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:07.634026 2500 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:07.635195 2500 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:07.637255 2502 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 0717a0d0c74f4a80b28209f66e3ca654 Changed from: seqno: 0 local: true peers { permanent_uuid: "0717a0d0c74f4a80b28209f66e3ca654" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "0717a0d0c74f4a80b28209f66e3ca654" role: LEADER } I1102 12:55:07.637334 2502 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "0717a0d0c74f4a80b28209f66e3ca654" role: LEADER } I1102 12:55:07.637964 2500 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:07.640894 2500 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:07.641806 2500 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:07.642793 2500 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:07.643177 2506 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 0717a0d0c74f4a80b28209f66e3ca654 Changed from: seqno: 0 local: true peers { permanent_uuid: "0717a0d0c74f4a80b28209f66e3ca654" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "0717a0d0c74f4a80b28209f66e3ca654" role: LEADER } I1102 12:55:07.643228 2506 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "0717a0d0c74f4a80b28209f66e3ca654" role: LEADER } I1102 12:55:07.643236 2500 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:07.654021 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "531498cb480346c0843797c61456a3fd" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:07.654173 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root uuid: "531498cb480346c0843797c61456a3fd" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:07.655266 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 496778 I1102 12:55:07.660240 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:07.660261 31819 webserver.cc:117] Document root disabled I1102 12:55:07.660544 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:40382 I1102 12:55:07.660696 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:07.662498 2549 heartbeater.cc:227] Registering TS with master... I1102 12:55:07.662559 2549 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:07.662991 2479 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "531498cb480346c0843797c61456a3fd" instance_seqno: 1414961707659932 } with Master I1102 12:55:07.663085 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.0021889s I1102 12:55:07.666615 2481 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:59858: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:07.673163 2527 tablet_service.cc:264] Processing CreateTablet for tablet 452fa80dae3a4f8fb0eb2145daeae7c8 (table=client-testtb [id=2bcf3e36cc7743c0bb1d21b770bfcbbe]), range=["", "\x00\x00\x00\t"] I1102 12:55:07.673243 2528 tablet_service.cc:264] Processing CreateTablet for tablet 9154a74211e249cb9017efa709753d6e (table=client-testtb [id=2bcf3e36cc7743c0bb1d21b770bfcbbe]), range=["\x00\x00\x00\t", ""] I1102 12:55:07.674263 2527 ts_tablet_manager.cc:460] Registered tablet 452fa80dae3a4f8fb0eb2145daeae7c8 I1102 12:55:07.674291 2528 ts_tablet_manager.cc:460] Registered tablet 9154a74211e249cb9017efa709753d6e I1102 12:55:07.674556 2558 ts_tablet_manager.cc:325] Bootstrapping tablet: 452fa80dae3a4f8fb0eb2145daeae7c8 I1102 12:55:07.674754 2559 ts_tablet_manager.cc:325] Bootstrapping tablet: 9154a74211e249cb9017efa709753d6e I1102 12:55:07.675130 2558 tablet_bootstrap.cc:234] Tablet 452fa80dae3a4f8fb0eb2145daeae7c8: Bootstrap starting. I1102 12:55:07.675195 2559 tablet_bootstrap.cc:234] Tablet 9154a74211e249cb9017efa709753d6e: Bootstrap starting. I1102 12:55:07.677738 2558 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 452fa80dae3a4f8fb0eb2145daeae7c8 creating new one. I1102 12:55:07.677763 2559 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 9154a74211e249cb9017efa709753d6e creating new one. I1102 12:55:07.679054 2559 tablet_bootstrap.cc:234] Tablet 9154a74211e249cb9017efa709753d6e: No bootstrap required, opened a new log I1102 12:55:07.679086 2558 tablet_bootstrap.cc:234] Tablet 452fa80dae3a4f8fb0eb2145daeae7c8: No bootstrap required, opened a new log I1102 12:55:07.679291 2559 ts_tablet_manager.cc:330] Time spent tablet 9154a74211e249cb9017efa709753d6e bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:55:07.679317 2558 ts_tablet_manager.cc:330] Time spent tablet 452fa80dae3a4f8fb0eb2145daeae7c8 bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:07.680882 2558 ts_tablet_manager.cc:350] Time spent starting tablet 452fa80dae3a4f8fb0eb2145daeae7c8: real 0.002s user 0.001s sys 0.000s I1102 12:55:07.680886 2559 ts_tablet_manager.cc:350] Time spent starting tablet 9154a74211e249cb9017efa709753d6e: real 0.002s user 0.001s sys 0.000s I1102 12:55:07.681509 2563 tablet_peer.cc:213] Configuration changed for tablet: 452fa80dae3a4f8fb0eb2145daeae7c8 in peer: 531498cb480346c0843797c61456a3fd Changed from: seqno: -1 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 60575 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } I1102 12:55:07.681823 2562 tablet_peer.cc:213] Configuration changed for tablet: 9154a74211e249cb9017efa709753d6e in peer: 531498cb480346c0843797c61456a3fd Changed from: seqno: -1 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 60575 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } I1102 12:55:07.685133 2479 catalog_manager.cc:1127] Tablet: 452fa80dae3a4f8fb0eb2145daeae7c8 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } I1102 12:55:07.687597 2479 catalog_manager.cc:1127] Tablet: 9154a74211e249cb9017efa709753d6e reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } I1102 12:55:07.698129 2484 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:59858: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:07.706451 2529 tablet_service.cc:264] Processing CreateTablet for tablet f4a6632eb2df4a82a718edf5da8acf17 (table=client-testtb2 [id=34da21b6cdb3467ca69bc81b47729dd5]), range=["", ""] I1102 12:55:07.708042 2529 ts_tablet_manager.cc:460] Registered tablet f4a6632eb2df4a82a718edf5da8acf17 I1102 12:55:07.708169 2559 ts_tablet_manager.cc:325] Bootstrapping tablet: f4a6632eb2df4a82a718edf5da8acf17 I1102 12:55:07.708649 2559 tablet_bootstrap.cc:234] Tablet f4a6632eb2df4a82a718edf5da8acf17: Bootstrap starting. I1102 12:55:07.711212 2559 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: f4a6632eb2df4a82a718edf5da8acf17 creating new one. I1102 12:55:07.712352 2559 tablet_bootstrap.cc:234] Tablet f4a6632eb2df4a82a718edf5da8acf17: No bootstrap required, opened a new log I1102 12:55:07.712610 2559 ts_tablet_manager.cc:330] Time spent tablet f4a6632eb2df4a82a718edf5da8acf17 bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:55:07.713702 2559 ts_tablet_manager.cc:350] Time spent starting tablet f4a6632eb2df4a82a718edf5da8acf17: real 0.001s user 0.001s sys 0.000s I1102 12:55:07.713913 2571 tablet_peer.cc:213] Configuration changed for tablet: f4a6632eb2df4a82a718edf5da8acf17 in peer: 531498cb480346c0843797c61456a3fd Changed from: seqno: -1 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 60575 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } I1102 12:55:07.722627 2483 catalog_manager.cc:1127] Tablet: f4a6632eb2df4a82a718edf5da8acf17 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } I1102 12:55:07.730221 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:07.742879 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:07.742949 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:07.742992 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: f4a6632eb2df4a82a718edf5da8acf17 I1102 12:55:07.743017 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(f4a6632eb2df4a82a718edf5da8acf17) I1102 12:55:07.743033 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(f4a6632eb2df4a82a718edf5da8acf17) I1102 12:55:07.743047 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(f4a6632eb2df4a82a718edf5da8acf17) I1102 12:55:07.743552 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 452fa80dae3a4f8fb0eb2145daeae7c8 I1102 12:55:07.743572 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(452fa80dae3a4f8fb0eb2145daeae7c8) I1102 12:55:07.743584 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(452fa80dae3a4f8fb0eb2145daeae7c8) I1102 12:55:07.743594 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(452fa80dae3a4f8fb0eb2145daeae7c8) I1102 12:55:07.744084 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 9154a74211e249cb9017efa709753d6e I1102 12:55:07.744104 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(9154a74211e249cb9017efa709753d6e) I1102 12:55:07.744114 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(9154a74211e249cb9017efa709753d6e) I1102 12:55:07.744123 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(9154a74211e249cb9017efa709753d6e) I1102 12:55:07.747916 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:07.747937 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:07.747969 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:07.749024 31819 master.cc:124] Master@127.0.0.1:57938 shutting down... I1102 12:55:07.760754 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:07.761266 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:07.764228 31819 master.cc:127] Master@127.0.0.1:57938 shutdown complete. I1102 12:55:07.774818 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root uuid: "0717a0d0c74f4a80b28209f66e3ca654" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:07.775588 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 496778 I1102 12:55:07.778156 31819 webserver.cc:108] Starting webserver on 0.0.0.0:50716 I1102 12:55:07.778178 31819 webserver.cc:117] Document root disabled I1102 12:55:07.778375 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:50716 I1102 12:55:07.779350 2612 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:07.781283 2612 tablet_bootstrap.cc:452] Will attempt to recover log segment: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/11111111111111111111111111111111/wal-000000001 to: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/11111111111111111111111111111111.recovery/wal-000000001 I1102 12:55:07.781333 2612 tablet_bootstrap.cc:462] Moved log directory: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/11111111111111111111111111111111 to recovery directory: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/11111111111111111111111111111111.recovery I1102 12:55:07.784726 2612 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap replayed 1/1 log segments. I1102 12:55:07.785044 2612 tablet_bootstrap.cc:480] Renamed old recovery dir from: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/11111111111111111111111111111111.recovery to: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/11111111111111111111111111111111.recovery-1414961707785023 I1102 12:55:07.785127 2612 tablet_bootstrap.cc:488] Removed renamed recovery dir: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/11111111111111111111111111111111.recovery-1414961707785023 I1102 12:55:07.785151 2612 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap complete. I1102 12:55:07.786984 2614 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 0717a0d0c74f4a80b28209f66e3ca654 Changed from: seqno: 1 local: true peers { permanent_uuid: "0717a0d0c74f4a80b28209f66e3ca654" role: LEADER } Changed to: seqno: 2 local: true peers { permanent_uuid: "0717a0d0c74f4a80b28209f66e3ca654" role: LEADER } I1102 12:55:07.787034 2614 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 2 local: true peers { permanent_uuid: "0717a0d0c74f4a80b28209f66e3ca654" role: LEADER } I1102 12:55:07.787924 2612 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:07.791209 2612 tablet_bootstrap.cc:452] Will attempt to recover log segment: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/00000000000000000000000000000000/wal-000000001 to: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/00000000000000000000000000000000.recovery/wal-000000001 I1102 12:55:07.791255 2612 tablet_bootstrap.cc:462] Moved log directory: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/00000000000000000000000000000000 to recovery directory: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/00000000000000000000000000000000.recovery I1102 12:55:07.799542 2612 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap replayed 1/1 log segments. I1102 12:55:07.799887 2612 tablet_bootstrap.cc:480] Renamed old recovery dir from: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/00000000000000000000000000000000.recovery to: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/00000000000000000000000000000000.recovery-1414961707799869 I1102 12:55:07.799965 2612 tablet_bootstrap.cc:488] Removed renamed recovery dir: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/master-0-root/wals/00000000000000000000000000000000.recovery-1414961707799869 I1102 12:55:07.799986 2612 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap complete. I1102 12:55:07.801061 2612 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:07.801390 2618 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 0717a0d0c74f4a80b28209f66e3ca654 Changed from: seqno: 1 local: true peers { permanent_uuid: "0717a0d0c74f4a80b28209f66e3ca654" role: LEADER } Changed to: seqno: 2 local: true peers { permanent_uuid: "0717a0d0c74f4a80b28209f66e3ca654" role: LEADER } I1102 12:55:07.801444 2612 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:07.801448 2618 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 2 local: true peers { permanent_uuid: "0717a0d0c74f4a80b28209f66e3ca654" role: LEADER } I1102 12:55:07.802321 2612 catalog_manager.cc:119] Loaded metadata for table [id=2bcf3e36cc7743c0bb1d21b770bfcbbe] I1102 12:55:07.802471 2612 catalog_manager.cc:119] Loaded metadata for table [id=34da21b6cdb3467ca69bc81b47729dd5] I1102 12:55:07.803254 2612 catalog_manager.cc:180] Loaded metadata for tablet 452fa80dae3a4f8fb0eb2145daeae7c8 (table client-testtb [id=2bcf3e36cc7743c0bb1d21b770bfcbbe]) I1102 12:55:07.803387 2612 catalog_manager.cc:180] Loaded metadata for tablet 9154a74211e249cb9017efa709753d6e (table client-testtb [id=2bcf3e36cc7743c0bb1d21b770bfcbbe]) I1102 12:55:07.803508 2612 catalog_manager.cc:180] Loaded metadata for tablet f4a6632eb2df4a82a718edf5da8acf17 (table client-testtb2 [id=34da21b6cdb3467ca69bc81b47729dd5]) I1102 12:55:07.812170 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root uuid: "531498cb480346c0843797c61456a3fd" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:07.812965 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 496778 I1102 12:55:07.813154 31819 ts_tablet_manager.cc:296] Loading master block f4a6632eb2df4a82a718edf5da8acf17 I1102 12:55:07.813565 31819 ts_tablet_manager.cc:460] Registered tablet f4a6632eb2df4a82a718edf5da8acf17 I1102 12:55:07.813802 31819 ts_tablet_manager.cc:296] Loading master block 9154a74211e249cb9017efa709753d6e I1102 12:55:07.813840 2631 ts_tablet_manager.cc:325] Bootstrapping tablet: f4a6632eb2df4a82a718edf5da8acf17 I1102 12:55:07.814066 31819 ts_tablet_manager.cc:460] Registered tablet 9154a74211e249cb9017efa709753d6e I1102 12:55:07.814262 2631 tablet_bootstrap.cc:234] Tablet f4a6632eb2df4a82a718edf5da8acf17: Bootstrap starting. I1102 12:55:07.814286 31819 ts_tablet_manager.cc:296] Loading master block 452fa80dae3a4f8fb0eb2145daeae7c8 I1102 12:55:07.814318 2632 ts_tablet_manager.cc:325] Bootstrapping tablet: 9154a74211e249cb9017efa709753d6e I1102 12:55:07.814544 31819 ts_tablet_manager.cc:460] Registered tablet 452fa80dae3a4f8fb0eb2145daeae7c8 I1102 12:55:07.814704 2632 tablet_bootstrap.cc:234] Tablet 9154a74211e249cb9017efa709753d6e: Bootstrap starting. I1102 12:55:07.814791 2634 ts_tablet_manager.cc:325] Bootstrapping tablet: 452fa80dae3a4f8fb0eb2145daeae7c8 I1102 12:55:07.815374 2634 tablet_bootstrap.cc:234] Tablet 452fa80dae3a4f8fb0eb2145daeae7c8: Bootstrap starting. I1102 12:55:07.816627 2631 tablet_bootstrap.cc:452] Will attempt to recover log segment: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/f4a6632eb2df4a82a718edf5da8acf17/wal-000000001 to: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/f4a6632eb2df4a82a718edf5da8acf17.recovery/wal-000000001 I1102 12:55:07.816699 2631 tablet_bootstrap.cc:462] Moved log directory: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/f4a6632eb2df4a82a718edf5da8acf17 to recovery directory: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/f4a6632eb2df4a82a718edf5da8acf17.recovery I1102 12:55:07.817549 2632 tablet_bootstrap.cc:452] Will attempt to recover log segment: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/9154a74211e249cb9017efa709753d6e/wal-000000001 to: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/9154a74211e249cb9017efa709753d6e.recovery/wal-000000001 I1102 12:55:07.817585 2632 tablet_bootstrap.cc:462] Moved log directory: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/9154a74211e249cb9017efa709753d6e to recovery directory: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/9154a74211e249cb9017efa709753d6e.recovery I1102 12:55:07.818403 2634 tablet_bootstrap.cc:452] Will attempt to recover log segment: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/452fa80dae3a4f8fb0eb2145daeae7c8/wal-000000001 to: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/452fa80dae3a4f8fb0eb2145daeae7c8.recovery/wal-000000001 I1102 12:55:07.818423 2631 tablet_bootstrap.cc:234] Tablet f4a6632eb2df4a82a718edf5da8acf17: Bootstrap replayed 1/1 log segments. I1102 12:55:07.818428 2634 tablet_bootstrap.cc:462] Moved log directory: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/452fa80dae3a4f8fb0eb2145daeae7c8 to recovery directory: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/452fa80dae3a4f8fb0eb2145daeae7c8.recovery I1102 12:55:07.818711 2631 tablet_bootstrap.cc:480] Renamed old recovery dir from: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/f4a6632eb2df4a82a718edf5da8acf17.recovery to: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/f4a6632eb2df4a82a718edf5da8acf17.recovery-1414961707818693 I1102 12:55:07.818768 2631 tablet_bootstrap.cc:488] Removed renamed recovery dir: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/f4a6632eb2df4a82a718edf5da8acf17.recovery-1414961707818693 I1102 12:55:07.818783 2631 tablet_bootstrap.cc:234] Tablet f4a6632eb2df4a82a718edf5da8acf17: Bootstrap complete. I1102 12:55:07.818883 2632 tablet_bootstrap.cc:234] Tablet 9154a74211e249cb9017efa709753d6e: Bootstrap replayed 1/1 log segments. I1102 12:55:07.819041 2631 ts_tablet_manager.cc:330] Time spent tablet f4a6632eb2df4a82a718edf5da8acf17 bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:07.819172 2632 tablet_bootstrap.cc:480] Renamed old recovery dir from: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/9154a74211e249cb9017efa709753d6e.recovery to: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/9154a74211e249cb9017efa709753d6e.recovery-1414961707819147 I1102 12:55:07.819267 2632 tablet_bootstrap.cc:488] Removed renamed recovery dir: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/9154a74211e249cb9017efa709753d6e.recovery-1414961707819147 I1102 12:55:07.819283 2632 tablet_bootstrap.cc:234] Tablet 9154a74211e249cb9017efa709753d6e: Bootstrap complete. I1102 12:55:07.819679 2632 ts_tablet_manager.cc:330] Time spent tablet 9154a74211e249cb9017efa709753d6e bootstrap: real 0.005s user 0.004s sys 0.001s I1102 12:55:07.819690 2634 tablet_bootstrap.cc:234] Tablet 452fa80dae3a4f8fb0eb2145daeae7c8: Bootstrap replayed 1/1 log segments. I1102 12:55:07.819990 2631 ts_tablet_manager.cc:350] Time spent starting tablet f4a6632eb2df4a82a718edf5da8acf17: real 0.001s user 0.001s sys 0.000s I1102 12:55:07.820137 2634 tablet_bootstrap.cc:480] Renamed old recovery dir from: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/452fa80dae3a4f8fb0eb2145daeae7c8.recovery to: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/452fa80dae3a4f8fb0eb2145daeae7c8.recovery-1414961707820109 I1102 12:55:07.820212 2634 tablet_bootstrap.cc:488] Removed renamed recovery dir: /tmp/kudutest-1000/client-test.ClientTest.TestStaleLocations.1414961696528764-31819/minicluster-data/ts-0-root/wals/452fa80dae3a4f8fb0eb2145daeae7c8.recovery-1414961707820109 I1102 12:55:07.820225 2634 tablet_bootstrap.cc:234] Tablet 452fa80dae3a4f8fb0eb2145daeae7c8: Bootstrap complete. I1102 12:55:07.820441 2634 ts_tablet_manager.cc:330] Time spent tablet 452fa80dae3a4f8fb0eb2145daeae7c8 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:07.820431 2652 tablet_peer.cc:213] Configuration changed for tablet: f4a6632eb2df4a82a718edf5da8acf17 in peer: 531498cb480346c0843797c61456a3fd Changed from: seqno: 0 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } Changed to: seqno: 1 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } I1102 12:55:07.820919 2632 ts_tablet_manager.cc:350] Time spent starting tablet 9154a74211e249cb9017efa709753d6e: real 0.001s user 0.000s sys 0.000s I1102 12:55:07.821154 2655 tablet_peer.cc:213] Configuration changed for tablet: 9154a74211e249cb9017efa709753d6e in peer: 531498cb480346c0843797c61456a3fd Changed from: seqno: 0 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } Changed to: seqno: 1 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } I1102 12:55:07.821238 2634 ts_tablet_manager.cc:350] Time spent starting tablet 452fa80dae3a4f8fb0eb2145daeae7c8: real 0.001s user 0.000s sys 0.000s I1102 12:55:07.821538 2658 tablet_peer.cc:213] Configuration changed for tablet: 452fa80dae3a4f8fb0eb2145daeae7c8 in peer: 531498cb480346c0843797c61456a3fd Changed from: seqno: 0 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } Changed to: seqno: 1 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } I1102 12:55:07.822412 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:07.822425 31819 webserver.cc:117] Document root disabled I1102 12:55:07.822670 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:55833 I1102 12:55:07.822803 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:07.824106 2684 heartbeater.cc:227] Registering TS with master... I1102 12:55:07.824164 2684 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:07.824589 2588 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "531498cb480346c0843797c61456a3fd" instance_seqno: 1414961707822158 } with Master I1102 12:55:07.824779 2588 catalog_manager.cc:1127] Tablet: f4a6632eb2df4a82a718edf5da8acf17 reported quorum change. New quorum: seqno: 1 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } I1102 12:55:07.825139 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00216734s I1102 12:55:07.825284 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:07.826254 2588 catalog_manager.cc:1127] Tablet: 452fa80dae3a4f8fb0eb2145daeae7c8 reported quorum change. New quorum: seqno: 1 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } I1102 12:55:07.827280 2588 catalog_manager.cc:1127] Tablet: 9154a74211e249cb9017efa709753d6e reported quorum change. New quorum: seqno: 1 local: true peers { permanent_uuid: "531498cb480346c0843797c61456a3fd" role: LEADER last_known_addr { host: "127.0.0.1" port: 60575 } } I1102 12:55:07.841115 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:07.841202 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:07.841253 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: f4a6632eb2df4a82a718edf5da8acf17 I1102 12:55:07.841286 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(f4a6632eb2df4a82a718edf5da8acf17) I1102 12:55:07.841310 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(f4a6632eb2df4a82a718edf5da8acf17) I1102 12:55:07.841325 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(f4a6632eb2df4a82a718edf5da8acf17) I1102 12:55:07.841815 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 452fa80dae3a4f8fb0eb2145daeae7c8 I1102 12:55:07.841836 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(452fa80dae3a4f8fb0eb2145daeae7c8) I1102 12:55:07.841846 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(452fa80dae3a4f8fb0eb2145daeae7c8) I1102 12:55:07.841856 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(452fa80dae3a4f8fb0eb2145daeae7c8) I1102 12:55:07.842190 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 9154a74211e249cb9017efa709753d6e I1102 12:55:07.842207 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(9154a74211e249cb9017efa709753d6e) I1102 12:55:07.842216 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(9154a74211e249cb9017efa709753d6e) I1102 12:55:07.842224 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(9154a74211e249cb9017efa709753d6e) I1102 12:55:07.845809 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:07.845834 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:07.845862 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:07.846760 31819 master.cc:124] Master@127.0.0.1:57938 shutting down... I1102 12:55:07.858396 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:07.858932 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:07.859371 31819 master.cc:127] Master@127.0.0.1:57938 shutdown complete. [ OK ] ClientTest.TestStaleLocations (393 ms) [ RUN ] ClientTest.TestReplicatedMultiTabletTable I1102 12:55:08.108907 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTable.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "ce44a374337c40bdae93a4fd442809c5" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:08.109066 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTable.1414961696528764-31819/minicluster-data/master-0-root uuid: "ce44a374337c40bdae93a4fd442809c5" format_stamp: "Formatted at 2014-11-02 20:55:07 on qiwi" I1102 12:55:08.110044 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 497278 I1102 12:55:08.114403 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:08.114445 31819 webserver.cc:117] Document root disabled I1102 12:55:08.114992 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:44467 I1102 12:55:08.118427 2716 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:08.122306 2716 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:08.123216 2716 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:08.124694 2718 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: ce44a374337c40bdae93a4fd442809c5 Changed from: seqno: 0 local: true peers { permanent_uuid: "ce44a374337c40bdae93a4fd442809c5" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "ce44a374337c40bdae93a4fd442809c5" role: LEADER } I1102 12:55:08.124759 2718 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "ce44a374337c40bdae93a4fd442809c5" role: LEADER } I1102 12:55:08.125226 2716 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:08.127300 2716 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:08.128168 2716 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:08.129232 2716 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:08.129585 2722 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: ce44a374337c40bdae93a4fd442809c5 Changed from: seqno: 0 local: true peers { permanent_uuid: "ce44a374337c40bdae93a4fd442809c5" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "ce44a374337c40bdae93a4fd442809c5" role: LEADER } I1102 12:55:08.129638 2716 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:08.129648 2722 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "ce44a374337c40bdae93a4fd442809c5" role: LEADER } I1102 12:55:08.142418 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTable.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" format_stamp: "Formatted at 2014-11-02 20:55:08 on qiwi" I1102 12:55:08.142518 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTable.1414961696528764-31819/minicluster-data/ts-0-root uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" format_stamp: "Formatted at 2014-11-02 20:55:08 on qiwi" I1102 12:55:08.143410 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 497278 I1102 12:55:08.147840 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:08.147860 31819 webserver.cc:117] Document root disabled I1102 12:55:08.148092 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:47615 I1102 12:55:08.148226 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:08.149822 2766 heartbeater.cc:227] Registering TS with master... I1102 12:55:08.149879 2766 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:08.150259 2693 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" instance_seqno: 1414961708147554 } with Master I1102 12:55:08.150519 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00216513s I1102 12:55:08.153571 2695 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:33010: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:08.159610 2744 tablet_service.cc:264] Processing CreateTablet for tablet 864cd20dd70645a58ca0e7dd329277dc (table=client-testtb [id=89ece73be203403d94584b2ac857d46e]), range=["", "\x00\x00\x00\t"] I1102 12:55:08.159661 2745 tablet_service.cc:264] Processing CreateTablet for tablet 0ccad0cf77534f5f8f81babb82fe270c (table=client-testtb [id=89ece73be203403d94584b2ac857d46e]), range=["\x00\x00\x00\t", ""] I1102 12:55:08.160614 2745 ts_tablet_manager.cc:460] Registered tablet 0ccad0cf77534f5f8f81babb82fe270c I1102 12:55:08.160698 2744 ts_tablet_manager.cc:460] Registered tablet 864cd20dd70645a58ca0e7dd329277dc I1102 12:55:08.161147 2776 ts_tablet_manager.cc:325] Bootstrapping tablet: 864cd20dd70645a58ca0e7dd329277dc I1102 12:55:08.161147 2775 ts_tablet_manager.cc:325] Bootstrapping tablet: 0ccad0cf77534f5f8f81babb82fe270c I1102 12:55:08.161869 2775 tablet_bootstrap.cc:234] Tablet 0ccad0cf77534f5f8f81babb82fe270c: Bootstrap starting. I1102 12:55:08.161869 2776 tablet_bootstrap.cc:234] Tablet 864cd20dd70645a58ca0e7dd329277dc: Bootstrap starting. I1102 12:55:08.165846 2775 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 0ccad0cf77534f5f8f81babb82fe270c creating new one. I1102 12:55:08.165860 2776 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 864cd20dd70645a58ca0e7dd329277dc creating new one. I1102 12:55:08.167426 2776 tablet_bootstrap.cc:234] Tablet 864cd20dd70645a58ca0e7dd329277dc: No bootstrap required, opened a new log I1102 12:55:08.167542 2775 tablet_bootstrap.cc:234] Tablet 0ccad0cf77534f5f8f81babb82fe270c: No bootstrap required, opened a new log I1102 12:55:08.167783 2776 ts_tablet_manager.cc:330] Time spent tablet 864cd20dd70645a58ca0e7dd329277dc bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:55:08.167922 2775 ts_tablet_manager.cc:330] Time spent tablet 0ccad0cf77534f5f8f81babb82fe270c bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:55:08.169266 2776 ts_tablet_manager.cc:350] Time spent starting tablet 864cd20dd70645a58ca0e7dd329277dc: real 0.001s user 0.001s sys 0.000s I1102 12:55:08.169270 2775 ts_tablet_manager.cc:350] Time spent starting tablet 0ccad0cf77534f5f8f81babb82fe270c: real 0.001s user 0.001s sys 0.000s I1102 12:55:08.169845 2780 tablet_peer.cc:213] Configuration changed for tablet: 0ccad0cf77534f5f8f81babb82fe270c in peer: 5cc3a713bea9447fa0a35a7a5dc8b2a0 Changed from: seqno: -1 local: true peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 53010 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: LEADER last_known_addr { host: "127.0.0.1" port: 53010 } } I1102 12:55:08.169848 2779 tablet_peer.cc:213] Configuration changed for tablet: 864cd20dd70645a58ca0e7dd329277dc in peer: 5cc3a713bea9447fa0a35a7a5dc8b2a0 Changed from: seqno: -1 local: true peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 53010 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: LEADER last_known_addr { host: "127.0.0.1" port: 53010 } } I1102 12:55:08.172098 2693 catalog_manager.cc:1127] Tablet: 864cd20dd70645a58ca0e7dd329277dc reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: LEADER last_known_addr { host: "127.0.0.1" port: 53010 } } I1102 12:55:08.173691 2693 catalog_manager.cc:1127] Tablet: 0ccad0cf77534f5f8f81babb82fe270c reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: LEADER last_known_addr { host: "127.0.0.1" port: 53010 } } I1102 12:55:08.182660 2698 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:33010: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:08.188345 2746 tablet_service.cc:264] Processing CreateTablet for tablet ee48e4f85aaf43358ba9f69f19525884 (table=client-testtb2 [id=1c97602a3cfe4fbeacd82cc9ca4c7e1e]), range=["", ""] I1102 12:55:08.189558 2746 ts_tablet_manager.cc:460] Registered tablet ee48e4f85aaf43358ba9f69f19525884 I1102 12:55:08.189659 2776 ts_tablet_manager.cc:325] Bootstrapping tablet: ee48e4f85aaf43358ba9f69f19525884 I1102 12:55:08.190143 2776 tablet_bootstrap.cc:234] Tablet ee48e4f85aaf43358ba9f69f19525884: Bootstrap starting. I1102 12:55:08.192661 2776 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: ee48e4f85aaf43358ba9f69f19525884 creating new one. I1102 12:55:08.193784 2776 tablet_bootstrap.cc:234] Tablet ee48e4f85aaf43358ba9f69f19525884: No bootstrap required, opened a new log I1102 12:55:08.194036 2776 ts_tablet_manager.cc:330] Time spent tablet ee48e4f85aaf43358ba9f69f19525884 bootstrap: real 0.004s user 0.003s sys 0.001s I1102 12:55:08.195224 2776 ts_tablet_manager.cc:350] Time spent starting tablet ee48e4f85aaf43358ba9f69f19525884: real 0.001s user 0.001s sys 0.000s I1102 12:55:08.195564 2786 tablet_peer.cc:213] Configuration changed for tablet: ee48e4f85aaf43358ba9f69f19525884 in peer: 5cc3a713bea9447fa0a35a7a5dc8b2a0 Changed from: seqno: -1 local: true peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 53010 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: LEADER last_known_addr { host: "127.0.0.1" port: 53010 } } I1102 12:55:08.196833 2693 catalog_manager.cc:1127] Tablet: ee48e4f85aaf43358ba9f69f19525884 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: LEADER last_known_addr { host: "127.0.0.1" port: 53010 } } I1102 12:55:08.217854 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTable.1414961696528764-31819/minicluster-data/ts-1-root/instance: uuid: "e37f8fd99e1f44da8424883fbb28a8c5" format_stamp: "Formatted at 2014-11-02 20:55:08 on qiwi" I1102 12:55:08.217985 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTable.1414961696528764-31819/minicluster-data/ts-1-root uuid: "e37f8fd99e1f44da8424883fbb28a8c5" format_stamp: "Formatted at 2014-11-02 20:55:08 on qiwi" I1102 12:55:08.218895 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 497278 I1102 12:55:08.227794 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:08.227820 31819 webserver.cc:117] Document root disabled I1102 12:55:08.228129 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:57575 I1102 12:55:08.228518 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:08.231425 2832 heartbeater.cc:227] Registering TS with master... I1102 12:55:08.231525 2832 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:08.232213 2694 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" instance_seqno: 1414961708227414 } with Master I1102 12:55:08.248033 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTable.1414961696528764-31819/minicluster-data/ts-2-root/instance: uuid: "0900f974a933481bb7ef88a95ea6b7bb" format_stamp: "Formatted at 2014-11-02 20:55:08 on qiwi" I1102 12:55:08.248127 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTable.1414961696528764-31819/minicluster-data/ts-2-root uuid: "0900f974a933481bb7ef88a95ea6b7bb" format_stamp: "Formatted at 2014-11-02 20:55:08 on qiwi" I1102 12:55:08.249392 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 497278 I1102 12:55:08.257905 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:08.257933 31819 webserver.cc:117] Document root disabled I1102 12:55:08.258190 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:44076 I1102 12:55:08.258574 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:08.260207 2874 heartbeater.cc:227] Registering TS with master... I1102 12:55:08.260259 2874 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:08.260579 2698 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" instance_seqno: 1414961708257574 } with Master I1102 12:55:08.261270 31819 mini_cluster.cc:268] 3 TS(s) registered with Master after 0.00221706s I1102 12:55:08.261898 2701 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:33010: name: "replicated" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 3 I1102 12:55:08.266774 2747 tablet_service.cc:264] Processing CreateTablet for tablet d19c0c638581472db098a03367e5d21d (table=replicated [id=25f20809543948c3b71b2b2644b31b5e]), range=["\x00\x00\x00\t", ""] I1102 12:55:08.266999 2748 tablet_service.cc:264] Processing CreateTablet for tablet 8285cc49c29f424bb9c536dec332c9f5 (table=replicated [id=25f20809543948c3b71b2b2644b31b5e]), range=["", "\x00\x00\x00\t"] I1102 12:55:08.267729 2809 tablet_service.cc:264] Processing CreateTablet for tablet 8285cc49c29f424bb9c536dec332c9f5 (table=replicated [id=25f20809543948c3b71b2b2644b31b5e]), range=["", "\x00\x00\x00\t"] I1102 12:55:08.267729 2808 tablet_service.cc:264] Processing CreateTablet for tablet d19c0c638581472db098a03367e5d21d (table=replicated [id=25f20809543948c3b71b2b2644b31b5e]), range=["\x00\x00\x00\t", ""] I1102 12:55:08.268169 2748 ts_tablet_manager.cc:460] Registered tablet 8285cc49c29f424bb9c536dec332c9f5 I1102 12:55:08.268235 2775 ts_tablet_manager.cc:325] Bootstrapping tablet: 8285cc49c29f424bb9c536dec332c9f5 I1102 12:55:08.268249 2747 ts_tablet_manager.cc:460] Registered tablet d19c0c638581472db098a03367e5d21d I1102 12:55:08.268307 2776 ts_tablet_manager.cc:325] Bootstrapping tablet: d19c0c638581472db098a03367e5d21d I1102 12:55:08.268405 2852 tablet_service.cc:264] Processing CreateTablet for tablet d19c0c638581472db098a03367e5d21d (table=replicated [id=25f20809543948c3b71b2b2644b31b5e]), range=["\x00\x00\x00\t", ""] I1102 12:55:08.268492 2853 tablet_service.cc:264] Processing CreateTablet for tablet 8285cc49c29f424bb9c536dec332c9f5 (table=replicated [id=25f20809543948c3b71b2b2644b31b5e]), range=["", "\x00\x00\x00\t"] I1102 12:55:08.268828 2776 tablet_bootstrap.cc:234] Tablet d19c0c638581472db098a03367e5d21d: Bootstrap starting. I1102 12:55:08.268931 2809 ts_tablet_manager.cc:460] Registered tablet 8285cc49c29f424bb9c536dec332c9f5 I1102 12:55:08.268985 2808 ts_tablet_manager.cc:460] Registered tablet d19c0c638581472db098a03367e5d21d I1102 12:55:08.269055 2775 tablet_bootstrap.cc:234] Tablet 8285cc49c29f424bb9c536dec332c9f5: Bootstrap starting. I1102 12:55:08.269556 2878 ts_tablet_manager.cc:325] Bootstrapping tablet: 8285cc49c29f424bb9c536dec332c9f5 I1102 12:55:08.269680 2852 ts_tablet_manager.cc:460] Registered tablet d19c0c638581472db098a03367e5d21d I1102 12:55:08.270117 2853 ts_tablet_manager.cc:460] Registered tablet 8285cc49c29f424bb9c536dec332c9f5 I1102 12:55:08.271276 2880 ts_tablet_manager.cc:325] Bootstrapping tablet: d19c0c638581472db098a03367e5d21d I1102 12:55:08.272081 2881 ts_tablet_manager.cc:325] Bootstrapping tablet: d19c0c638581472db098a03367e5d21d I1102 12:55:08.272371 2878 tablet_bootstrap.cc:234] Tablet 8285cc49c29f424bb9c536dec332c9f5: Bootstrap starting. I1102 12:55:08.272377 2879 ts_tablet_manager.cc:325] Bootstrapping tablet: 8285cc49c29f424bb9c536dec332c9f5 I1102 12:55:08.272899 2881 tablet_bootstrap.cc:234] Tablet d19c0c638581472db098a03367e5d21d: Bootstrap starting. I1102 12:55:08.273149 2880 tablet_bootstrap.cc:234] Tablet d19c0c638581472db098a03367e5d21d: Bootstrap starting. I1102 12:55:08.274500 2879 tablet_bootstrap.cc:234] Tablet 8285cc49c29f424bb9c536dec332c9f5: Bootstrap starting. I1102 12:55:08.275686 2775 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 8285cc49c29f424bb9c536dec332c9f5 creating new one. I1102 12:55:08.277201 2776 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: d19c0c638581472db098a03367e5d21d creating new one. I1102 12:55:08.277654 2775 tablet_bootstrap.cc:234] Tablet 8285cc49c29f424bb9c536dec332c9f5: No bootstrap required, opened a new log I1102 12:55:08.277777 2881 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: d19c0c638581472db098a03367e5d21d creating new one. I1102 12:55:08.277883 2775 ts_tablet_manager.cc:330] Time spent tablet 8285cc49c29f424bb9c536dec332c9f5 bootstrap: real 0.010s user 0.005s sys 0.002s I1102 12:55:08.278818 2880 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: d19c0c638581472db098a03367e5d21d creating new one. I1102 12:55:08.278956 2878 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 8285cc49c29f424bb9c536dec332c9f5 creating new one. I1102 12:55:08.279182 2881 tablet_bootstrap.cc:234] Tablet d19c0c638581472db098a03367e5d21d: No bootstrap required, opened a new log I1102 12:55:08.279476 2776 tablet_bootstrap.cc:234] Tablet d19c0c638581472db098a03367e5d21d: No bootstrap required, opened a new log I1102 12:55:08.279625 2881 ts_tablet_manager.cc:330] Time spent tablet d19c0c638581472db098a03367e5d21d bootstrap: real 0.007s user 0.005s sys 0.001s I1102 12:55:08.279866 2776 ts_tablet_manager.cc:330] Time spent tablet d19c0c638581472db098a03367e5d21d bootstrap: real 0.012s user 0.006s sys 0.002s I1102 12:55:08.280207 2775 raft_consensus.cc:227] T 8285cc49c29f424bb9c536dec332c9f5 P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [LEADER]: Becoming Leader I1102 12:55:08.280335 2880 tablet_bootstrap.cc:234] Tablet d19c0c638581472db098a03367e5d21d: No bootstrap required, opened a new log I1102 12:55:08.280408 2879 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 8285cc49c29f424bb9c536dec332c9f5 creating new one. I1102 12:55:08.280585 2880 ts_tablet_manager.cc:330] Time spent tablet d19c0c638581472db098a03367e5d21d bootstrap: real 0.009s user 0.006s sys 0.001s I1102 12:55:08.280597 2881 raft_consensus.cc:284] T d19c0c638581472db098a03367e5d21d P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Becoming Follower/Learner I1102 12:55:08.280773 2776 raft_consensus.cc:284] T d19c0c638581472db098a03367e5d21d P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [FOLLOWER]: Becoming Follower/Learner I1102 12:55:08.280803 2881 ts_tablet_manager.cc:350] Time spent starting tablet d19c0c638581472db098a03367e5d21d: real 0.001s user 0.001s sys 0.000s I1102 12:55:08.280905 2776 ts_tablet_manager.cc:350] Time spent starting tablet d19c0c638581472db098a03367e5d21d: real 0.001s user 0.001s sys 0.000s I1102 12:55:08.281219 2878 tablet_bootstrap.cc:234] Tablet 8285cc49c29f424bb9c536dec332c9f5: No bootstrap required, opened a new log I1102 12:55:08.281441 2878 ts_tablet_manager.cc:330] Time spent tablet 8285cc49c29f424bb9c536dec332c9f5 bootstrap: real 0.012s user 0.006s sys 0.002s I1102 12:55:08.282107 2880 raft_consensus.cc:227] T d19c0c638581472db098a03367e5d21d P e37f8fd99e1f44da8424883fbb28a8c5 [LEADER]: Becoming Leader I1102 12:55:08.282449 2878 raft_consensus.cc:284] T 8285cc49c29f424bb9c536dec332c9f5 P e37f8fd99e1f44da8424883fbb28a8c5 [FOLLOWER]: Becoming Follower/Learner I1102 12:55:08.282567 2878 ts_tablet_manager.cc:350] Time spent starting tablet 8285cc49c29f424bb9c536dec332c9f5: real 0.001s user 0.001s sys 0.000s I1102 12:55:08.282764 2879 tablet_bootstrap.cc:234] Tablet 8285cc49c29f424bb9c536dec332c9f5: No bootstrap required, opened a new log I1102 12:55:08.282982 2879 ts_tablet_manager.cc:330] Time spent tablet 8285cc49c29f424bb9c536dec332c9f5 bootstrap: real 0.010s user 0.005s sys 0.002s I1102 12:55:08.283807 2879 raft_consensus.cc:284] T 8285cc49c29f424bb9c536dec332c9f5 P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Becoming Follower/Learner I1102 12:55:08.283928 2879 ts_tablet_manager.cc:350] Time spent starting tablet 8285cc49c29f424bb9c536dec332c9f5: real 0.001s user 0.001s sys 0.000s I1102 12:55:08.285341 2880 ts_tablet_manager.cc:350] Time spent starting tablet d19c0c638581472db098a03367e5d21d: real 0.005s user 0.002s sys 0.001s I1102 12:55:08.285593 2775 ts_tablet_manager.cc:350] Time spent starting tablet 8285cc49c29f424bb9c536dec332c9f5: real 0.008s user 0.004s sys 0.001s W1102 12:55:08.782021 2894 raft_consensus_state.cc:462] T 8285cc49c29f424bb9c536dec332c9f5 P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [LEADER]: Can't advance the committed index across term boundaries yet. Last committed operation was: term: 0 index: 0 New majority replicated is: term: 0 index: 0. Current term is: 1 W1102 12:55:08.782631 2883 raft_consensus_state.cc:462] T 8285cc49c29f424bb9c536dec332c9f5 P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [LEADER]: Can't advance the committed index across term boundaries yet. Last committed operation was: term: 0 index: 0 New majority replicated is: term: 0 index: 0. Current term is: 1 W1102 12:55:08.783110 2897 raft_consensus_state.cc:462] T d19c0c638581472db098a03367e5d21d P e37f8fd99e1f44da8424883fbb28a8c5 [LEADER]: Can't advance the committed index across term boundaries yet. Last committed operation was: term: 0 index: 0 New majority replicated is: term: 0 index: 0. Current term is: 1 W1102 12:55:08.783540 2890 raft_consensus_state.cc:462] T d19c0c638581472db098a03367e5d21d P e37f8fd99e1f44da8424883fbb28a8c5 [LEADER]: Can't advance the committed index across term boundaries yet. Last committed operation was: term: 0 index: 0 New majority replicated is: term: 0 index: 0. Current term is: 1 I1102 12:55:08.785291 2754 raft_consensus.cc:582] T d19c0c638581472db098a03367e5d21d P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [FOLLOWER]: Refusing update from remote peer: Log matching property violated. Last replicated by replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. I1102 12:55:08.785317 2820 raft_consensus.cc:582] T 8285cc49c29f424bb9c536dec332c9f5 P e37f8fd99e1f44da8424883fbb28a8c5 [FOLLOWER]: Refusing update from remote peer: Log matching property violated. Last replicated by replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. I1102 12:55:08.785403 2863 raft_consensus.cc:582] T d19c0c638581472db098a03367e5d21d P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Refusing update from remote peer: Log matching property violated. Last replicated by replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. I1102 12:55:08.785429 2862 raft_consensus.cc:582] T 8285cc49c29f424bb9c536dec332c9f5 P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Refusing update from remote peer: Log matching property violated. Last replicated by replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. I1102 12:55:08.785640 2790 consensus_queue.cc:453] Connected to peer: 0900f974a933481bb7ef88a95ea6b7bb whose last received watermark is: term: 0 index: 0 I1102 12:55:08.785704 2728 consensus_queue.cc:453] Connected to peer: 0900f974a933481bb7ef88a95ea6b7bb whose last received watermark is: term: 0 index: 0 I1102 12:55:08.785807 2791 consensus_queue.cc:453] Connected to peer: 5cc3a713bea9447fa0a35a7a5dc8b2a0 whose last received watermark is: term: 0 index: 0 I1102 12:55:08.786202 2727 consensus_queue.cc:453] Connected to peer: e37f8fd99e1f44da8424883fbb28a8c5 whose last received watermark is: term: 0 index: 0 I1102 12:55:08.788103 2728 raft_consensus_state.cc:455] T 8285cc49c29f424bb9c536dec332c9f5 P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [LEADER]: Advanced the committed_index across terms. Last committed operation was: term: 0 index: 0 New committed index is: term: 1 index: 1 I1102 12:55:08.788254 2791 raft_consensus_state.cc:455] T d19c0c638581472db098a03367e5d21d P e37f8fd99e1f44da8424883fbb28a8c5 [LEADER]: Advanced the committed_index across terms. Last committed operation was: term: 0 index: 0 New committed index is: term: 1 index: 1 I1102 12:55:08.788365 2883 tablet_peer.cc:213] Configuration changed for tablet: 8285cc49c29f424bb9c536dec332c9f5 in peer: 5cc3a713bea9447fa0a35a7a5dc8b2a0 Changed from: seqno: -1 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: LEADER last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } I1102 12:55:08.788489 2890 tablet_peer.cc:213] Configuration changed for tablet: d19c0c638581472db098a03367e5d21d in peer: e37f8fd99e1f44da8424883fbb28a8c5 Changed from: seqno: -1 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: LEADER last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } I1102 12:55:08.796715 2696 catalog_manager.cc:1127] Tablet: 8285cc49c29f424bb9c536dec332c9f5 reported quorum change. New quorum: seqno: 0 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: LEADER last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } I1102 12:55:08.797201 2697 catalog_manager.cc:1127] Tablet: d19c0c638581472db098a03367e5d21d reported quorum change. New quorum: seqno: 0 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: LEADER last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } I1102 12:55:08.823806 2891 tablet_peer.cc:213] Configuration changed for tablet: 8285cc49c29f424bb9c536dec332c9f5 in peer: e37f8fd99e1f44da8424883fbb28a8c5 Changed from: seqno: -1 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: LEADER last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } I1102 12:55:08.824069 2896 tablet_peer.cc:213] Configuration changed for tablet: 8285cc49c29f424bb9c536dec332c9f5 in peer: 0900f974a933481bb7ef88a95ea6b7bb Changed from: seqno: -1 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: LEADER last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } I1102 12:55:08.825716 2886 tablet_peer.cc:213] Configuration changed for tablet: d19c0c638581472db098a03367e5d21d in peer: 0900f974a933481bb7ef88a95ea6b7bb Changed from: seqno: -1 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: LEADER last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } I1102 12:55:08.825834 2887 tablet_peer.cc:213] Configuration changed for tablet: d19c0c638581472db098a03367e5d21d in peer: 5cc3a713bea9447fa0a35a7a5dc8b2a0 Changed from: seqno: -1 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "e37f8fd99e1f44da8424883fbb28a8c5" role: LEADER last_known_addr { host: "127.0.0.1" port: 47481 } } peers { permanent_uuid: "5cc3a713bea9447fa0a35a7a5dc8b2a0" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 53010 } } peers { permanent_uuid: "0900f974a933481bb7ef88a95ea6b7bb" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 48822 } } I1102 12:55:08.836381 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:08.849460 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:08.849524 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:08.849577 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 864cd20dd70645a58ca0e7dd329277dc I1102 12:55:08.849617 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(864cd20dd70645a58ca0e7dd329277dc) I1102 12:55:08.849645 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(864cd20dd70645a58ca0e7dd329277dc) I1102 12:55:08.849669 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(864cd20dd70645a58ca0e7dd329277dc) I1102 12:55:08.850297 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: d19c0c638581472db098a03367e5d21d I1102 12:55:08.850335 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(d19c0c638581472db098a03367e5d21d) I1102 12:55:08.850364 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(d19c0c638581472db098a03367e5d21d) I1102 12:55:08.850389 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(d19c0c638581472db098a03367e5d21d) I1102 12:55:08.850430 31819 raft_consensus.cc:769] T d19c0c638581472db098a03367e5d21d P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [FOLLOWER]: Raft consensus shutting down. I1102 12:55:08.850491 31819 raft_consensus_state.cc:377] T d19c0c638581472db098a03367e5d21d P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [FOLLOWER]: Trying to abort 1 pending transactions. I1102 12:55:08.850533 31819 raft_consensus_state.cc:383] T d19c0c638581472db098a03367e5d21d P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [FOLLOWER]: Aborting transaction as it isn't in flight: id { term: 1 index: 2 } op_type: WRITE_OP write_request { tablet_id: "d19c0c638581472db098a03367e5d21d" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } row_operations { rows: "\001\017\000\t\000\000\000\022\000\000\000\000\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\033\000\000\000\001\017\000\n\000\000\000\024\000\000\000\007\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000\036\000\000\000\001\017\000\013\000\000\000\026\000\000\000\017\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000!\000\000\000\001\017\000\014\000\000\000\030\000\000\000\027\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000$\000\000\000\001\017\000\r\000\000\000\032\000\000\000\037\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000\'\000\000\000\001\017\000\016\000\000\000\034\000\000\000\'\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000*\000\000\000\001\017\000\017\000\000\000\036\000\000\000/\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000-\000\000\000\001\017\000\020\000\000\000 \000\000\0007\000\000\000\000\000\000\000\010\000\000\000\000\000\000\0000\000\000\000\001\017\000\021\000\000\000\"\000\000\000?\000\000\000\000\000\000\000\010\000\000\000\000\000\000\0003\000\000\000\001\017\000\022\000\000\000$\000\000\000G\000\000\000\000\000\000\000\010\000\000\000\000\000\000\0006\000\000\000\001\017\000\023\000\000\000&\000\000\000O\000\000\000\000\000\000\000\010\000\000\000\000\000\000\0009\000\000\000\001\017\000\024\000\000\000(\000\000\000W\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000<\000\000\000\001\017\000\025\000\000\000*\000\000\000_\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000?\000\000\000\001\017\000\026\000\000\000,\000\000\000g\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000B\000\000\000\001\017\000\027\000\000\000.\000\000\000o\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000E\000\000\000\001\017\000\030\000\000\0000\000\000\000w\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000H\000\000\000\001\017\000\031\000\000\0002\000\000\000\177\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000K\000\000\000\001\017\000\032\000\000\0004\000\000\000\207\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000N\000\000\000\001\017\000\033\000\000\0006\000\000\000\217\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000Q\000\000\000\001\017\000\034\000\000\0008\000\000\000\227\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000T\000\000\000\001\017\000\035\000\000\000:\000\000\000\237\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000W\000\000\000\001\017\000\036\000\000\000<\000\000\000\247\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000Z\000\000\000\001\017\000\037\000\000\000>\000\000\000\257\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000]\000\000\000\001\017\000 \000\000\000@\000\000\000\267\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000`\000\000\000\001\017\000!\000\000\000B\000\000\000\277\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000c\000\000\000\001\017\000\"\000\000\000D\000\000\000\307\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000f\000\000\000\001\017\000#\000\000\000F\000\000\000\317\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000i\000\000\000\001\017\000$\000\000\000H\000\000\000\327\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000l\000\000\000\001\017\000%\000\000\000J\000\000\000\337\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000o\000\000\000\001\017\000&\000\000\000L\000\000\000\347\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000r\000\000\000\001\017\000\'\000\000\000N\000\000\000\357\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000u\000\000\000\001\017\000(\000\000\000P\000\000\000\367\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000x\000\000\000\001\017\000)\000\000\000R\000\000\000\377\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000{\000\000\000\001\017\000*\000\000\000T\000\000\000\007\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000~\000\000\000\001\017\000+\000\000\000V\000\000\000\017\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\201\000\000\000\001\017\000,\000\000\000X\000\000\000\027\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\204\000\000\000\001\017\000-\000\000\000Z\000\000\000\037\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\207\000\000\000\001\017\000.\000\000\000\\\000\000\000\'\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\212\000\000\000\001\017\000/\000\000\000^\000\000\000/\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\215\000\000\000\001\017\0000\000\000\000`\000\000\0007\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\220\000\000\000\001\017\0001\000\000\000b\000\000\000?\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\223\000\000\000\001\017\0002\000\000\000d\000\000\000G\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\226\000\000\000\001\017\0003\000\000\000f\000\000\000O\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\231\000\000\000\001\017\0004\000\000\000h\000\000\000W\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\234\000\000\000\001\017\0005\000\000\000j\000\000\000_\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\237\000\000\000\001\017\0006\000\000\000l\000\000\000g\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\242\000\000\000\001\017\0007\000\000\000n\000\000\000o\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\245\000\000\000\001\017\0008\000\000\000p\000\000\000w\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\250\000\000\000\001\017\0009\000\000\000r\000\000\000\177\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\253\000\000\000\001\017\000:\000\000\000t\000\000\000\207\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\256\000\000\000\001\017\000;\000\000\000v\000\000\000\217\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\261\000\000\000\001\017\000<\000\000\000x\000\000\000\227\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\264\000\000\000\001\017\000=\000\000\000z\000\000\000\237\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\267\000\000\000\001\017\000>\000\000\000|\000\000\000\247\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\272\000\000\000\001\017\000?\000\000\000~\000\000\000\257\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\275\000\000\000\001\017\000@\000\000\000\200\000\000\000\267\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\300\000\000\000\001\017\000A\000\000\000\202\000\000\000\277\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\303\000\000\000\001\017\000B\000\000\000\204\000\000\000\307\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\306\000\000\000\001\017\000C\000\000\000\206\000\000\000\317\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\311\000\000\000\001\017\000D\000\000\000\210\000\000\000\327\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\314\000\000\000\001\017\000E\000\000\000\212\000\000\000\337\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\317\000\000\000\001\017\000F\000\000\000\214\000\000\000\347\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\322\000\000\000\001\017\000G\000\000\000\216\000\000\000\357\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\325\000\000\000\001\017\000H\000\000\000\220\000\000\000\367\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\330\000\000\000\001\017\000I\000\000\000\222\000\000\000\377\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\333\000\000\000\001\017\000J\000\000\000\224\000\000\000\007\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\336\000\000\000\001\017\000K\000\000\000\226\000\000\000\017\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\341\000\000\000\001\017\000L\000\000\000\230\000\000\000\027\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\344\000\000\000\001\017\000M\000\000\000\232\000\000\000\037\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\347\000\000\000\001\017\000N\000\000\000\234\000\000\000\'\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\352\000\000\000\001\017\000O\000\000\000\236\000\000\000/\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\355\000\000\000\001\017\000P\000\000\000\240\000\000\0007\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\360\000\000\000\001\017\000Q\000\000\000\242\000\000\000?\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\363\000\000\000\001\017\000R\000\000\000\244\000\000\000G\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\366\000\000\000\001\017\000S\000\000\000\246\000\000\000O\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\371\000\000\000\001\017\000T\000\000\000\250\000\000\000W\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\374\000\000\000\001\017\000U\000\000\000\252\000\000\000_\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\377\000\000\000\001\017\000V\000\000\000\254\000\000\000g\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\002\001\000\000\001\017\000W\000\000\000\256\000\000\000o\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\005\001\000\000\001\017\000X\000\000\000\260\000\000\000w\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\010\001\000\000\001\017\000Y\000\000\000\262\000\000\000\177\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\013\001\000\000\001\017\000Z\000\000\000\264\000\000\000\207\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\016\001\000\000\001\017\000[\000\000\000\266\000\000\000\217\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\021\001\000\000\001\017\000\\\000\000\000\270\000\000\000\227\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\024\001\000\000\001\017\000]\000\000\000\272\000\000\000\237\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\027\001\000\000\001\017\000^\000\000\000\274\000\000\000\247\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\032\001\000\000\001\017\000_\000\000\000\276\000\000\000\257\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\035\001\000\000\001\017\000`\000\000\000\300\000\000\000\267\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000 \001\000\000\001\017\000a\000\000\000\302\000\000\000\277\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000#\001\000\000\001\017\000b\000\000\000\304\000\000\000\307\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000&\001\000\000\001\017\000c\000\000\000\306\000\000\000\317\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000)\001\000\000" indirect_data: "hello 9hello 10hello 11hello 12hello 13hello 14hello 15hello 16hello 17hello 18hello 19hello 20hello 21hello 22hello 23hello 24hello 25hello 26hello 27hello 28hello 29hello 30hello 31hello 32hello 33hello 34hello 35hello 36hello 37hello 38hello 39hello 40hello 41hello 42hello 43hello 44hello 45hello 46hello 47hello 48hello 49hello 50hello 51hello 52hello 53hello 54hello 55hello 56hello 57hello 58hello 59hello 60hello 61hello 62hello 63hello 64hello 65hello 66hello 67hello 68hello 69hello 70hello 71hello 72hello 73hello 74hello 75hello 76hello 77hello 78hello 79hello 80hello 81hello 82hello 83hello 84hello 85hello 86hello 87hello 88hello 89hello 90hello 91hello 92hello 93hello 94hello 95hello 96hello 97hello 98hello 99" } } I1102 12:55:08.852092 31819 raft_consensus_state.cc:404] T d19c0c638581472db098a03367e5d21d P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [FOLLOWER]: Waiting on 0 outstanding applies: I1102 12:55:08.852128 31819 raft_consensus_state.cc:408] T d19c0c638581472db098a03367e5d21d P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [FOLLOWER]: All local commits completed. I1102 12:55:08.852166 31819 raft_consensus.cc:789] T d19c0c638581472db098a03367e5d21d P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [FOLLOWER]: Raft consensus Shutdown! I1102 12:55:08.852674 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 8285cc49c29f424bb9c536dec332c9f5 I1102 12:55:08.852711 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(8285cc49c29f424bb9c536dec332c9f5) I1102 12:55:08.852741 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(8285cc49c29f424bb9c536dec332c9f5) I1102 12:55:08.852764 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(8285cc49c29f424bb9c536dec332c9f5) I1102 12:55:08.852794 31819 raft_consensus.cc:769] T 8285cc49c29f424bb9c536dec332c9f5 P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [LEADER]: Raft consensus shutting down. I1102 12:55:08.852957 31819 consensus_peers.cc:387] Closing peer: 5cc3a713bea9447fa0a35a7a5dc8b2a0 I1102 12:55:08.853144 31819 consensus_peers.cc:387] Closing peer: 0900f974a933481bb7ef88a95ea6b7bb I1102 12:55:08.853250 31819 consensus_peers.cc:387] Closing peer: e37f8fd99e1f44da8424883fbb28a8c5 I1102 12:55:08.853282 31819 raft_consensus_state.cc:377] T 8285cc49c29f424bb9c536dec332c9f5 P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [LEADER]: Trying to abort 0 pending transactions. I1102 12:55:08.853302 31819 raft_consensus_state.cc:404] T 8285cc49c29f424bb9c536dec332c9f5 P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [LEADER]: Waiting on 0 outstanding applies: I1102 12:55:08.853324 31819 raft_consensus_state.cc:408] T 8285cc49c29f424bb9c536dec332c9f5 P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [LEADER]: All local commits completed. I1102 12:55:08.853345 31819 raft_consensus.cc:789] T 8285cc49c29f424bb9c536dec332c9f5 P 5cc3a713bea9447fa0a35a7a5dc8b2a0 [LEADER]: Raft consensus Shutdown! I1102 12:55:08.853689 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: ee48e4f85aaf43358ba9f69f19525884 I1102 12:55:08.853715 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(ee48e4f85aaf43358ba9f69f19525884) I1102 12:55:08.853735 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(ee48e4f85aaf43358ba9f69f19525884) I1102 12:55:08.853752 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(ee48e4f85aaf43358ba9f69f19525884) I1102 12:55:08.854012 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 0ccad0cf77534f5f8f81babb82fe270c I1102 12:55:08.854037 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(0ccad0cf77534f5f8f81babb82fe270c) I1102 12:55:08.854056 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(0ccad0cf77534f5f8f81babb82fe270c) I1102 12:55:08.854071 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(0ccad0cf77534f5f8f81babb82fe270c) I1102 12:55:08.864327 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:08.864356 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:08.864398 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:08.865427 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:08.882917 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:08.883059 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:08.883180 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: d19c0c638581472db098a03367e5d21d I1102 12:55:08.883272 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(d19c0c638581472db098a03367e5d21d) I1102 12:55:08.883352 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(d19c0c638581472db098a03367e5d21d) I1102 12:55:08.883414 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(d19c0c638581472db098a03367e5d21d) I1102 12:55:08.883510 31819 raft_consensus.cc:769] T d19c0c638581472db098a03367e5d21d P e37f8fd99e1f44da8424883fbb28a8c5 [LEADER]: Raft consensus shutting down. I1102 12:55:08.883934 31819 consensus_peers.cc:387] Closing peer: 5cc3a713bea9447fa0a35a7a5dc8b2a0 I1102 12:55:08.884321 31819 consensus_peers.cc:387] Closing peer: 0900f974a933481bb7ef88a95ea6b7bb I1102 12:55:08.884742 31819 consensus_peers.cc:387] Closing peer: e37f8fd99e1f44da8424883fbb28a8c5 I1102 12:55:08.884861 31819 raft_consensus_state.cc:377] T d19c0c638581472db098a03367e5d21d P e37f8fd99e1f44da8424883fbb28a8c5 [LEADER]: Trying to abort 0 pending transactions. I1102 12:55:08.884944 31819 raft_consensus_state.cc:404] T d19c0c638581472db098a03367e5d21d P e37f8fd99e1f44da8424883fbb28a8c5 [LEADER]: Waiting on 0 outstanding applies: I1102 12:55:08.885025 31819 raft_consensus_state.cc:408] T d19c0c638581472db098a03367e5d21d P e37f8fd99e1f44da8424883fbb28a8c5 [LEADER]: All local commits completed. I1102 12:55:08.885162 31819 raft_consensus.cc:789] T d19c0c638581472db098a03367e5d21d P e37f8fd99e1f44da8424883fbb28a8c5 [LEADER]: Raft consensus Shutdown! I1102 12:55:08.886550 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 8285cc49c29f424bb9c536dec332c9f5 I1102 12:55:08.886652 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(8285cc49c29f424bb9c536dec332c9f5) I1102 12:55:08.886723 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(8285cc49c29f424bb9c536dec332c9f5) I1102 12:55:08.886782 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(8285cc49c29f424bb9c536dec332c9f5) I1102 12:55:08.886869 31819 raft_consensus.cc:769] T 8285cc49c29f424bb9c536dec332c9f5 P e37f8fd99e1f44da8424883fbb28a8c5 [FOLLOWER]: Raft consensus shutting down. I1102 12:55:08.886988 31819 raft_consensus_state.cc:377] T 8285cc49c29f424bb9c536dec332c9f5 P e37f8fd99e1f44da8424883fbb28a8c5 [FOLLOWER]: Trying to abort 1 pending transactions. I1102 12:55:08.887055 31819 raft_consensus_state.cc:383] T 8285cc49c29f424bb9c536dec332c9f5 P e37f8fd99e1f44da8424883fbb28a8c5 [FOLLOWER]: Aborting transaction as it isn't in flight: id { term: 1 index: 2 } op_type: WRITE_OP write_request { tablet_id: "8285cc49c29f424bb9c536dec332c9f5" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } row_operations { rows: "\001\017\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\000\000\000\000\001\017\000\001\000\000\000\002\000\000\000\007\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\003\000\000\000\001\017\000\002\000\000\000\004\000\000\000\016\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\006\000\000\000\001\017\000\003\000\000\000\006\000\000\000\025\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\t\000\000\000\001\017\000\004\000\000\000\010\000\000\000\034\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\014\000\000\000\001\017\000\005\000\000\000\n\000\000\000#\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\017\000\000\000\001\017\000\006\000\000\000\014\000\000\000*\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\022\000\000\000\001\017\000\007\000\000\000\016\000\000\0001\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\025\000\000\000\001\017\000\010\000\000\000\020\000\000\0008\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\030\000\000\000" indirect_data: "hello 0hello 1hello 2hello 3hello 4hello 5hello 6hello 7hello 8" } } I1102 12:55:08.888387 31819 raft_consensus_state.cc:404] T 8285cc49c29f424bb9c536dec332c9f5 P e37f8fd99e1f44da8424883fbb28a8c5 [FOLLOWER]: Waiting on 0 outstanding applies: I1102 12:55:08.888473 31819 raft_consensus_state.cc:408] T 8285cc49c29f424bb9c536dec332c9f5 P e37f8fd99e1f44da8424883fbb28a8c5 [FOLLOWER]: All local commits completed. I1102 12:55:08.888525 31819 raft_consensus.cc:789] T 8285cc49c29f424bb9c536dec332c9f5 P e37f8fd99e1f44da8424883fbb28a8c5 [FOLLOWER]: Raft consensus Shutdown! I1102 12:55:08.893092 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:08.893146 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:08.893187 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:08.894112 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:08.909175 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:08.909235 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:08.909286 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 8285cc49c29f424bb9c536dec332c9f5 I1102 12:55:08.909322 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(8285cc49c29f424bb9c536dec332c9f5) I1102 12:55:08.909345 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(8285cc49c29f424bb9c536dec332c9f5) I1102 12:55:08.909364 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(8285cc49c29f424bb9c536dec332c9f5) I1102 12:55:08.909400 31819 raft_consensus.cc:769] T 8285cc49c29f424bb9c536dec332c9f5 P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Raft consensus shutting down. I1102 12:55:08.909463 31819 raft_consensus_state.cc:377] T 8285cc49c29f424bb9c536dec332c9f5 P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Trying to abort 1 pending transactions. I1102 12:55:08.909497 31819 raft_consensus_state.cc:383] T 8285cc49c29f424bb9c536dec332c9f5 P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Aborting transaction as it isn't in flight: id { term: 1 index: 2 } op_type: WRITE_OP write_request { tablet_id: "8285cc49c29f424bb9c536dec332c9f5" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } row_operations { rows: "\001\017\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\000\000\000\000\001\017\000\001\000\000\000\002\000\000\000\007\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\003\000\000\000\001\017\000\002\000\000\000\004\000\000\000\016\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\006\000\000\000\001\017\000\003\000\000\000\006\000\000\000\025\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\t\000\000\000\001\017\000\004\000\000\000\010\000\000\000\034\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\014\000\000\000\001\017\000\005\000\000\000\n\000\000\000#\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\017\000\000\000\001\017\000\006\000\000\000\014\000\000\000*\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\022\000\000\000\001\017\000\007\000\000\000\016\000\000\0001\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\025\000\000\000\001\017\000\010\000\000\000\020\000\000\0008\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\030\000\000\000" indirect_data: "hello 0hello 1hello 2hello 3hello 4hello 5hello 6hello 7hello 8" } } I1102 12:55:08.910089 31819 raft_consensus_state.cc:404] T 8285cc49c29f424bb9c536dec332c9f5 P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Waiting on 0 outstanding applies: I1102 12:55:08.910135 31819 raft_consensus_state.cc:408] T 8285cc49c29f424bb9c536dec332c9f5 P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: All local commits completed. I1102 12:55:08.910166 31819 raft_consensus.cc:789] T 8285cc49c29f424bb9c536dec332c9f5 P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Raft consensus Shutdown! I1102 12:55:08.910912 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: d19c0c638581472db098a03367e5d21d I1102 12:55:08.910955 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(d19c0c638581472db098a03367e5d21d) I1102 12:55:08.910979 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(d19c0c638581472db098a03367e5d21d) I1102 12:55:08.910997 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(d19c0c638581472db098a03367e5d21d) I1102 12:55:08.911030 31819 raft_consensus.cc:769] T d19c0c638581472db098a03367e5d21d P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Raft consensus shutting down. I1102 12:55:08.911077 31819 raft_consensus_state.cc:377] T d19c0c638581472db098a03367e5d21d P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Trying to abort 1 pending transactions. I1102 12:55:08.911106 31819 raft_consensus_state.cc:383] T d19c0c638581472db098a03367e5d21d P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Aborting transaction as it isn't in flight: id { term: 1 index: 2 } op_type: WRITE_OP write_request { tablet_id: "d19c0c638581472db098a03367e5d21d" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } row_operations { rows: "\001\017\000\t\000\000\000\022\000\000\000\000\000\000\000\000\000\000\000\007\000\000\000\000\000\000\000\033\000\000\000\001\017\000\n\000\000\000\024\000\000\000\007\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000\036\000\000\000\001\017\000\013\000\000\000\026\000\000\000\017\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000!\000\000\000\001\017\000\014\000\000\000\030\000\000\000\027\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000$\000\000\000\001\017\000\r\000\000\000\032\000\000\000\037\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000\'\000\000\000\001\017\000\016\000\000\000\034\000\000\000\'\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000*\000\000\000\001\017\000\017\000\000\000\036\000\000\000/\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000-\000\000\000\001\017\000\020\000\000\000 \000\000\0007\000\000\000\000\000\000\000\010\000\000\000\000\000\000\0000\000\000\000\001\017\000\021\000\000\000\"\000\000\000?\000\000\000\000\000\000\000\010\000\000\000\000\000\000\0003\000\000\000\001\017\000\022\000\000\000$\000\000\000G\000\000\000\000\000\000\000\010\000\000\000\000\000\000\0006\000\000\000\001\017\000\023\000\000\000&\000\000\000O\000\000\000\000\000\000\000\010\000\000\000\000\000\000\0009\000\000\000\001\017\000\024\000\000\000(\000\000\000W\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000<\000\000\000\001\017\000\025\000\000\000*\000\000\000_\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000?\000\000\000\001\017\000\026\000\000\000,\000\000\000g\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000B\000\000\000\001\017\000\027\000\000\000.\000\000\000o\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000E\000\000\000\001\017\000\030\000\000\0000\000\000\000w\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000H\000\000\000\001\017\000\031\000\000\0002\000\000\000\177\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000K\000\000\000\001\017\000\032\000\000\0004\000\000\000\207\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000N\000\000\000\001\017\000\033\000\000\0006\000\000\000\217\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000Q\000\000\000\001\017\000\034\000\000\0008\000\000\000\227\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000T\000\000\000\001\017\000\035\000\000\000:\000\000\000\237\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000W\000\000\000\001\017\000\036\000\000\000<\000\000\000\247\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000Z\000\000\000\001\017\000\037\000\000\000>\000\000\000\257\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000]\000\000\000\001\017\000 \000\000\000@\000\000\000\267\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000`\000\000\000\001\017\000!\000\000\000B\000\000\000\277\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000c\000\000\000\001\017\000\"\000\000\000D\000\000\000\307\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000f\000\000\000\001\017\000#\000\000\000F\000\000\000\317\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000i\000\000\000\001\017\000$\000\000\000H\000\000\000\327\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000l\000\000\000\001\017\000%\000\000\000J\000\000\000\337\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000o\000\000\000\001\017\000&\000\000\000L\000\000\000\347\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000r\000\000\000\001\017\000\'\000\000\000N\000\000\000\357\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000u\000\000\000\001\017\000(\000\000\000P\000\000\000\367\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000x\000\000\000\001\017\000)\000\000\000R\000\000\000\377\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000{\000\000\000\001\017\000*\000\000\000T\000\000\000\007\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000~\000\000\000\001\017\000+\000\000\000V\000\000\000\017\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\201\000\000\000\001\017\000,\000\000\000X\000\000\000\027\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\204\000\000\000\001\017\000-\000\000\000Z\000\000\000\037\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\207\000\000\000\001\017\000.\000\000\000\\\000\000\000\'\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\212\000\000\000\001\017\000/\000\000\000^\000\000\000/\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\215\000\000\000\001\017\0000\000\000\000`\000\000\0007\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\220\000\000\000\001\017\0001\000\000\000b\000\000\000?\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\223\000\000\000\001\017\0002\000\000\000d\000\000\000G\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\226\000\000\000\001\017\0003\000\000\000f\000\000\000O\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\231\000\000\000\001\017\0004\000\000\000h\000\000\000W\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\234\000\000\000\001\017\0005\000\000\000j\000\000\000_\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\237\000\000\000\001\017\0006\000\000\000l\000\000\000g\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\242\000\000\000\001\017\0007\000\000\000n\000\000\000o\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\245\000\000\000\001\017\0008\000\000\000p\000\000\000w\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\250\000\000\000\001\017\0009\000\000\000r\000\000\000\177\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\253\000\000\000\001\017\000:\000\000\000t\000\000\000\207\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\256\000\000\000\001\017\000;\000\000\000v\000\000\000\217\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\261\000\000\000\001\017\000<\000\000\000x\000\000\000\227\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\264\000\000\000\001\017\000=\000\000\000z\000\000\000\237\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\267\000\000\000\001\017\000>\000\000\000|\000\000\000\247\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\272\000\000\000\001\017\000?\000\000\000~\000\000\000\257\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\275\000\000\000\001\017\000@\000\000\000\200\000\000\000\267\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\300\000\000\000\001\017\000A\000\000\000\202\000\000\000\277\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\303\000\000\000\001\017\000B\000\000\000\204\000\000\000\307\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\306\000\000\000\001\017\000C\000\000\000\206\000\000\000\317\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\311\000\000\000\001\017\000D\000\000\000\210\000\000\000\327\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\314\000\000\000\001\017\000E\000\000\000\212\000\000\000\337\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\317\000\000\000\001\017\000F\000\000\000\214\000\000\000\347\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\322\000\000\000\001\017\000G\000\000\000\216\000\000\000\357\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\325\000\000\000\001\017\000H\000\000\000\220\000\000\000\367\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\330\000\000\000\001\017\000I\000\000\000\222\000\000\000\377\001\000\000\000\000\000\000\010\000\000\000\000\000\000\000\333\000\000\000\001\017\000J\000\000\000\224\000\000\000\007\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\336\000\000\000\001\017\000K\000\000\000\226\000\000\000\017\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\341\000\000\000\001\017\000L\000\000\000\230\000\000\000\027\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\344\000\000\000\001\017\000M\000\000\000\232\000\000\000\037\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\347\000\000\000\001\017\000N\000\000\000\234\000\000\000\'\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\352\000\000\000\001\017\000O\000\000\000\236\000\000\000/\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\355\000\000\000\001\017\000P\000\000\000\240\000\000\0007\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\360\000\000\000\001\017\000Q\000\000\000\242\000\000\000?\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\363\000\000\000\001\017\000R\000\000\000\244\000\000\000G\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\366\000\000\000\001\017\000S\000\000\000\246\000\000\000O\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\371\000\000\000\001\017\000T\000\000\000\250\000\000\000W\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\374\000\000\000\001\017\000U\000\000\000\252\000\000\000_\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\377\000\000\000\001\017\000V\000\000\000\254\000\000\000g\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\002\001\000\000\001\017\000W\000\000\000\256\000\000\000o\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\005\001\000\000\001\017\000X\000\000\000\260\000\000\000w\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\010\001\000\000\001\017\000Y\000\000\000\262\000\000\000\177\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\013\001\000\000\001\017\000Z\000\000\000\264\000\000\000\207\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\016\001\000\000\001\017\000[\000\000\000\266\000\000\000\217\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\021\001\000\000\001\017\000\\\000\000\000\270\000\000\000\227\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\024\001\000\000\001\017\000]\000\000\000\272\000\000\000\237\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\027\001\000\000\001\017\000^\000\000\000\274\000\000\000\247\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\032\001\000\000\001\017\000_\000\000\000\276\000\000\000\257\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000\035\001\000\000\001\017\000`\000\000\000\300\000\000\000\267\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000 \001\000\000\001\017\000a\000\000\000\302\000\000\000\277\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000#\001\000\000\001\017\000b\000\000\000\304\000\000\000\307\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000&\001\000\000\001\017\000c\000\000\000\306\000\000\000\317\002\000\000\000\000\000\000\010\000\000\000\000\000\000\000)\001\000\000" indirect_data: "hello 9hello 10hello 11hello 12hello 13hello 14hello 15hello 16hello 17hello 18hello 19hello 20hello 21hello 22hello 23hello 24hello 25hello 26hello 27hello 28hello 29hello 30hello 31hello 32hello 33hello 34hello 35hello 36hello 37hello 38hello 39hello 40hello 41hello 42hello 43hello 44hello 45hello 46hello 47hello 48hello 49hello 50hello 51hello 52hello 53hello 54hello 55hello 56hello 57hello 58hello 59hello 60hello 61hello 62hello 63hello 64hello 65hello 66hello 67hello 68hello 69hello 70hello 71hello 72hello 73hello 74hello 75hello 76hello 77hello 78hello 79hello 80hello 81hello 82hello 83hello 84hello 85hello 86hello 87hello 88hello 89hello 90hello 91hello 92hello 93hello 94hello 95hello 96hello 97hello 98hello 99" } } I1102 12:55:08.913214 31819 raft_consensus_state.cc:404] T d19c0c638581472db098a03367e5d21d P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Waiting on 0 outstanding applies: I1102 12:55:08.913242 31819 raft_consensus_state.cc:408] T d19c0c638581472db098a03367e5d21d P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: All local commits completed. I1102 12:55:08.913261 31819 raft_consensus.cc:789] T d19c0c638581472db098a03367e5d21d P 0900f974a933481bb7ef88a95ea6b7bb [FOLLOWER]: Raft consensus Shutdown! I1102 12:55:08.916923 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:08.916944 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:08.916975 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:08.917870 31819 master.cc:124] Master@127.0.0.1:37578 shutting down... I1102 12:55:08.930973 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:08.931970 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:08.933084 31819 master.cc:127] Master@127.0.0.1:37578 shutdown complete. [ OK ] ClientTest.TestReplicatedMultiTabletTable (1082 ms) [ RUN ] ClientTest.TestReplicatedMultiTabletTableFailover I1102 12:55:09.228085 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTableFailover.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "35cb4ec4915347a695bac0452dde4c2a" format_stamp: "Formatted at 2014-11-02 20:55:08 on qiwi" I1102 12:55:09.228196 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTableFailover.1414961696528764-31819/minicluster-data/master-0-root uuid: "35cb4ec4915347a695bac0452dde4c2a" format_stamp: "Formatted at 2014-11-02 20:55:08 on qiwi" I1102 12:55:09.229050 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 497778 I1102 12:55:09.231734 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:09.231750 31819 webserver.cc:117] Document root disabled I1102 12:55:09.231942 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:41947 I1102 12:55:09.233194 2945 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:09.234999 2945 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:09.235824 2945 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:09.237175 2947 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 35cb4ec4915347a695bac0452dde4c2a Changed from: seqno: 0 local: true peers { permanent_uuid: "35cb4ec4915347a695bac0452dde4c2a" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "35cb4ec4915347a695bac0452dde4c2a" role: LEADER } I1102 12:55:09.237231 2947 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "35cb4ec4915347a695bac0452dde4c2a" role: LEADER } I1102 12:55:09.237685 2945 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:09.239697 2945 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:09.240545 2945 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:09.241578 2945 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:09.241984 2951 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 35cb4ec4915347a695bac0452dde4c2a Changed from: seqno: 0 local: true peers { permanent_uuid: "35cb4ec4915347a695bac0452dde4c2a" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "35cb4ec4915347a695bac0452dde4c2a" role: LEADER } I1102 12:55:09.242034 2945 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:09.242039 2951 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "35cb4ec4915347a695bac0452dde4c2a" role: LEADER } I1102 12:55:09.255370 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTableFailover.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "530d10cfdcdf43d891aca53a64d58a88" format_stamp: "Formatted at 2014-11-02 20:55:09 on qiwi" I1102 12:55:09.255523 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTableFailover.1414961696528764-31819/minicluster-data/ts-0-root uuid: "530d10cfdcdf43d891aca53a64d58a88" format_stamp: "Formatted at 2014-11-02 20:55:09 on qiwi" I1102 12:55:09.256598 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 497778 I1102 12:55:09.263298 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:09.263320 31819 webserver.cc:117] Document root disabled I1102 12:55:09.263633 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:50835 I1102 12:55:09.263854 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:09.265909 2994 heartbeater.cc:227] Registering TS with master... I1102 12:55:09.265985 2994 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:09.266481 2924 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" instance_seqno: 1414961709262878 } with Master I1102 12:55:09.267365 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00330097s I1102 12:55:09.271682 2926 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:41764: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:09.279762 2972 tablet_service.cc:264] Processing CreateTablet for tablet 4b8d89d654e04b96b5955951098ddfce (table=client-testtb [id=d534401acba04bd6843035f1d7bea425]), range=["", "\x00\x00\x00\t"] I1102 12:55:09.279826 2973 tablet_service.cc:264] Processing CreateTablet for tablet 7deb92afd493467fb4355ff7aa9c3b27 (table=client-testtb [id=d534401acba04bd6843035f1d7bea425]), range=["\x00\x00\x00\t", ""] I1102 12:55:09.281060 2973 ts_tablet_manager.cc:460] Registered tablet 7deb92afd493467fb4355ff7aa9c3b27 I1102 12:55:09.281108 2972 ts_tablet_manager.cc:460] Registered tablet 4b8d89d654e04b96b5955951098ddfce I1102 12:55:09.281388 3003 ts_tablet_manager.cc:325] Bootstrapping tablet: 7deb92afd493467fb4355ff7aa9c3b27 I1102 12:55:09.281661 3004 ts_tablet_manager.cc:325] Bootstrapping tablet: 4b8d89d654e04b96b5955951098ddfce I1102 12:55:09.281999 3003 tablet_bootstrap.cc:234] Tablet 7deb92afd493467fb4355ff7aa9c3b27: Bootstrap starting. I1102 12:55:09.282251 3004 tablet_bootstrap.cc:234] Tablet 4b8d89d654e04b96b5955951098ddfce: Bootstrap starting. I1102 12:55:09.285285 3003 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 7deb92afd493467fb4355ff7aa9c3b27 creating new one. I1102 12:55:09.285327 3004 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 4b8d89d654e04b96b5955951098ddfce creating new one. I1102 12:55:09.286936 3003 tablet_bootstrap.cc:234] Tablet 7deb92afd493467fb4355ff7aa9c3b27: No bootstrap required, opened a new log I1102 12:55:09.286978 3004 tablet_bootstrap.cc:234] Tablet 4b8d89d654e04b96b5955951098ddfce: No bootstrap required, opened a new log I1102 12:55:09.287242 3003 ts_tablet_manager.cc:330] Time spent tablet 7deb92afd493467fb4355ff7aa9c3b27 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:09.287272 3004 ts_tablet_manager.cc:330] Time spent tablet 4b8d89d654e04b96b5955951098ddfce bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:09.289038 3004 ts_tablet_manager.cc:350] Time spent starting tablet 4b8d89d654e04b96b5955951098ddfce: real 0.002s user 0.001s sys 0.001s I1102 12:55:09.289242 3003 ts_tablet_manager.cc:350] Time spent starting tablet 7deb92afd493467fb4355ff7aa9c3b27: real 0.002s user 0.001s sys 0.001s I1102 12:55:09.289970 3007 tablet_peer.cc:213] Configuration changed for tablet: 7deb92afd493467fb4355ff7aa9c3b27 in peer: 530d10cfdcdf43d891aca53a64d58a88 Changed from: seqno: -1 local: true peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 49414 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: LEADER last_known_addr { host: "127.0.0.1" port: 49414 } } I1102 12:55:09.289999 3008 tablet_peer.cc:213] Configuration changed for tablet: 4b8d89d654e04b96b5955951098ddfce in peer: 530d10cfdcdf43d891aca53a64d58a88 Changed from: seqno: -1 local: true peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 49414 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: LEADER last_known_addr { host: "127.0.0.1" port: 49414 } } I1102 12:55:09.299700 2926 catalog_manager.cc:1127] Tablet: 4b8d89d654e04b96b5955951098ddfce reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: LEADER last_known_addr { host: "127.0.0.1" port: 49414 } } I1102 12:55:09.302649 2926 catalog_manager.cc:1127] Tablet: 7deb92afd493467fb4355ff7aa9c3b27 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: LEADER last_known_addr { host: "127.0.0.1" port: 49414 } } I1102 12:55:09.312795 2931 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:41764: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:09.321765 2974 tablet_service.cc:264] Processing CreateTablet for tablet 150dc7143d7b4b539ab695ff7e1757b9 (table=client-testtb2 [id=9e5006c76ec44b1a93b6a994548ac51e]), range=["", ""] I1102 12:55:09.323278 2974 ts_tablet_manager.cc:460] Registered tablet 150dc7143d7b4b539ab695ff7e1757b9 I1102 12:55:09.323426 3004 ts_tablet_manager.cc:325] Bootstrapping tablet: 150dc7143d7b4b539ab695ff7e1757b9 I1102 12:55:09.324120 3004 tablet_bootstrap.cc:234] Tablet 150dc7143d7b4b539ab695ff7e1757b9: Bootstrap starting. I1102 12:55:09.327728 3004 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 150dc7143d7b4b539ab695ff7e1757b9 creating new one. I1102 12:55:09.329257 3004 tablet_bootstrap.cc:234] Tablet 150dc7143d7b4b539ab695ff7e1757b9: No bootstrap required, opened a new log I1102 12:55:09.329612 3004 ts_tablet_manager.cc:330] Time spent tablet 150dc7143d7b4b539ab695ff7e1757b9 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:09.331217 3004 ts_tablet_manager.cc:350] Time spent starting tablet 150dc7143d7b4b539ab695ff7e1757b9: real 0.002s user 0.001s sys 0.000s I1102 12:55:09.331701 3014 tablet_peer.cc:213] Configuration changed for tablet: 150dc7143d7b4b539ab695ff7e1757b9 in peer: 530d10cfdcdf43d891aca53a64d58a88 Changed from: seqno: -1 local: true peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 49414 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: LEADER last_known_addr { host: "127.0.0.1" port: 49414 } } I1102 12:55:09.338524 2930 catalog_manager.cc:1127] Tablet: 150dc7143d7b4b539ab695ff7e1757b9 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: LEADER last_known_addr { host: "127.0.0.1" port: 49414 } } I1102 12:55:09.366534 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTableFailover.1414961696528764-31819/minicluster-data/ts-1-root/instance: uuid: "eedd3ed575384fcf880ceb61df2e3cb6" format_stamp: "Formatted at 2014-11-02 20:55:09 on qiwi" I1102 12:55:09.366636 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTableFailover.1414961696528764-31819/minicluster-data/ts-1-root uuid: "eedd3ed575384fcf880ceb61df2e3cb6" format_stamp: "Formatted at 2014-11-02 20:55:09 on qiwi" I1102 12:55:09.367487 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 497778 I1102 12:55:09.374176 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:09.374204 31819 webserver.cc:117] Document root disabled I1102 12:55:09.374564 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:49083 I1102 12:55:09.374814 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:09.376898 3056 heartbeater.cc:227] Registering TS with master... I1102 12:55:09.376984 3056 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:09.377550 2929 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" instance_seqno: 1414961709373737 } with Master I1102 12:55:09.392098 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTableFailover.1414961696528764-31819/minicluster-data/ts-2-root/instance: uuid: "50fad9a13654438a99c95c9d534c73f1" format_stamp: "Formatted at 2014-11-02 20:55:09 on qiwi" I1102 12:55:09.392189 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedMultiTabletTableFailover.1414961696528764-31819/minicluster-data/ts-2-root uuid: "50fad9a13654438a99c95c9d534c73f1" format_stamp: "Formatted at 2014-11-02 20:55:09 on qiwi" I1102 12:55:09.393002 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 497778 I1102 12:55:09.396914 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:09.396934 31819 webserver.cc:117] Document root disabled I1102 12:55:09.397227 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:52821 I1102 12:55:09.397380 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:09.399226 3101 heartbeater.cc:227] Registering TS with master... I1102 12:55:09.399301 3101 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:09.399695 2925 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" instance_seqno: 1414961709396657 } with Master I1102 12:55:09.399827 31819 mini_cluster.cc:268] 3 TS(s) registered with Master after 0.00222101s I1102 12:55:09.400595 2927 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:41764: name: "replicated_failover_on_reads" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 3 I1102 12:55:09.407549 2975 tablet_service.cc:264] Processing CreateTablet for tablet 6cc27b921ae14cb4ae55c0401e4c58e9 (table=replicated_failover_on_reads [id=42449b55e32b4bc7a51bafcb82acf53e]), range=["\x00\x00\x00\t", ""] I1102 12:55:09.408061 2976 tablet_service.cc:264] Processing CreateTablet for tablet 4961d3da9c5d424f820aa4d6753fdeb6 (table=replicated_failover_on_reads [id=42449b55e32b4bc7a51bafcb82acf53e]), range=["", "\x00\x00\x00\t"] I1102 12:55:09.408218 3034 tablet_service.cc:264] Processing CreateTablet for tablet 6cc27b921ae14cb4ae55c0401e4c58e9 (table=replicated_failover_on_reads [id=42449b55e32b4bc7a51bafcb82acf53e]), range=["\x00\x00\x00\t", ""] I1102 12:55:09.408298 3035 tablet_service.cc:264] Processing CreateTablet for tablet 4961d3da9c5d424f820aa4d6753fdeb6 (table=replicated_failover_on_reads [id=42449b55e32b4bc7a51bafcb82acf53e]), range=["", "\x00\x00\x00\t"] I1102 12:55:09.408669 3078 tablet_service.cc:264] Processing CreateTablet for tablet 6cc27b921ae14cb4ae55c0401e4c58e9 (table=replicated_failover_on_reads [id=42449b55e32b4bc7a51bafcb82acf53e]), range=["\x00\x00\x00\t", ""] I1102 12:55:09.408759 3080 tablet_service.cc:264] Processing CreateTablet for tablet 4961d3da9c5d424f820aa4d6753fdeb6 (table=replicated_failover_on_reads [id=42449b55e32b4bc7a51bafcb82acf53e]), range=["", "\x00\x00\x00\t"] I1102 12:55:09.408984 2975 ts_tablet_manager.cc:460] Registered tablet 6cc27b921ae14cb4ae55c0401e4c58e9 I1102 12:55:09.409139 3003 ts_tablet_manager.cc:325] Bootstrapping tablet: 6cc27b921ae14cb4ae55c0401e4c58e9 I1102 12:55:09.409586 2976 ts_tablet_manager.cc:460] Registered tablet 4961d3da9c5d424f820aa4d6753fdeb6 I1102 12:55:09.409744 3004 ts_tablet_manager.cc:325] Bootstrapping tablet: 4961d3da9c5d424f820aa4d6753fdeb6 I1102 12:55:09.411131 3035 ts_tablet_manager.cc:460] Registered tablet 4961d3da9c5d424f820aa4d6753fdeb6 I1102 12:55:09.411276 3034 ts_tablet_manager.cc:460] Registered tablet 6cc27b921ae14cb4ae55c0401e4c58e9 I1102 12:55:09.411986 3078 ts_tablet_manager.cc:460] Registered tablet 6cc27b921ae14cb4ae55c0401e4c58e9 I1102 12:55:09.412320 3105 ts_tablet_manager.cc:325] Bootstrapping tablet: 4961d3da9c5d424f820aa4d6753fdeb6 I1102 12:55:09.412688 3003 tablet_bootstrap.cc:234] Tablet 6cc27b921ae14cb4ae55c0401e4c58e9: Bootstrap starting. I1102 12:55:09.412833 3080 ts_tablet_manager.cc:460] Registered tablet 4961d3da9c5d424f820aa4d6753fdeb6 I1102 12:55:09.413822 3004 tablet_bootstrap.cc:234] Tablet 4961d3da9c5d424f820aa4d6753fdeb6: Bootstrap starting. I1102 12:55:09.414173 3106 ts_tablet_manager.cc:325] Bootstrapping tablet: 6cc27b921ae14cb4ae55c0401e4c58e9 I1102 12:55:09.414173 3108 ts_tablet_manager.cc:325] Bootstrapping tablet: 4961d3da9c5d424f820aa4d6753fdeb6 I1102 12:55:09.414175 3109 ts_tablet_manager.cc:325] Bootstrapping tablet: 6cc27b921ae14cb4ae55c0401e4c58e9 I1102 12:55:09.415156 3105 tablet_bootstrap.cc:234] Tablet 4961d3da9c5d424f820aa4d6753fdeb6: Bootstrap starting. I1102 12:55:09.416096 3109 tablet_bootstrap.cc:234] Tablet 6cc27b921ae14cb4ae55c0401e4c58e9: Bootstrap starting. I1102 12:55:09.416173 3108 tablet_bootstrap.cc:234] Tablet 4961d3da9c5d424f820aa4d6753fdeb6: Bootstrap starting. I1102 12:55:09.416370 3106 tablet_bootstrap.cc:234] Tablet 6cc27b921ae14cb4ae55c0401e4c58e9: Bootstrap starting. I1102 12:55:09.418977 3004 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 4961d3da9c5d424f820aa4d6753fdeb6 creating new one. I1102 12:55:09.419234 3109 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 6cc27b921ae14cb4ae55c0401e4c58e9 creating new one. I1102 12:55:09.419697 3003 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 6cc27b921ae14cb4ae55c0401e4c58e9 creating new one. I1102 12:55:09.420148 3004 tablet_bootstrap.cc:234] Tablet 4961d3da9c5d424f820aa4d6753fdeb6: No bootstrap required, opened a new log I1102 12:55:09.420289 3109 tablet_bootstrap.cc:234] Tablet 6cc27b921ae14cb4ae55c0401e4c58e9: No bootstrap required, opened a new log I1102 12:55:09.420425 3105 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 4961d3da9c5d424f820aa4d6753fdeb6 creating new one. I1102 12:55:09.420558 3004 ts_tablet_manager.cc:330] Time spent tablet 4961d3da9c5d424f820aa4d6753fdeb6 bootstrap: real 0.011s user 0.006s sys 0.001s I1102 12:55:09.420755 3109 ts_tablet_manager.cc:330] Time spent tablet 6cc27b921ae14cb4ae55c0401e4c58e9 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:09.421644 3108 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 4961d3da9c5d424f820aa4d6753fdeb6 creating new one. I1102 12:55:09.421880 3004 raft_consensus.cc:284] T 4961d3da9c5d424f820aa4d6753fdeb6 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: Becoming Follower/Learner I1102 12:55:09.421964 3106 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 6cc27b921ae14cb4ae55c0401e4c58e9 creating new one. I1102 12:55:09.422075 3004 ts_tablet_manager.cc:350] Time spent starting tablet 4961d3da9c5d424f820aa4d6753fdeb6: real 0.001s user 0.001s sys 0.000s I1102 12:55:09.422250 3105 tablet_bootstrap.cc:234] Tablet 4961d3da9c5d424f820aa4d6753fdeb6: No bootstrap required, opened a new log I1102 12:55:09.422443 3003 tablet_bootstrap.cc:234] Tablet 6cc27b921ae14cb4ae55c0401e4c58e9: No bootstrap required, opened a new log I1102 12:55:09.422480 3109 raft_consensus.cc:227] T 6cc27b921ae14cb4ae55c0401e4c58e9 P eedd3ed575384fcf880ceb61df2e3cb6 [LEADER]: Becoming Leader I1102 12:55:09.422593 3105 ts_tablet_manager.cc:330] Time spent tablet 4961d3da9c5d424f820aa4d6753fdeb6 bootstrap: real 0.010s user 0.006s sys 0.002s I1102 12:55:09.422755 3003 ts_tablet_manager.cc:330] Time spent tablet 6cc27b921ae14cb4ae55c0401e4c58e9 bootstrap: real 0.014s user 0.007s sys 0.002s I1102 12:55:09.423146 3106 tablet_bootstrap.cc:234] Tablet 6cc27b921ae14cb4ae55c0401e4c58e9: No bootstrap required, opened a new log I1102 12:55:09.423248 3105 raft_consensus.cc:284] T 4961d3da9c5d424f820aa4d6753fdeb6 P eedd3ed575384fcf880ceb61df2e3cb6 [FOLLOWER]: Becoming Follower/Learner I1102 12:55:09.423360 3105 ts_tablet_manager.cc:350] Time spent starting tablet 4961d3da9c5d424f820aa4d6753fdeb6: real 0.001s user 0.001s sys 0.000s I1102 12:55:09.423519 3106 ts_tablet_manager.cc:330] Time spent tablet 6cc27b921ae14cb4ae55c0401e4c58e9 bootstrap: real 0.009s user 0.006s sys 0.001s I1102 12:55:09.423542 3003 raft_consensus.cc:284] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: Becoming Follower/Learner I1102 12:55:09.423656 3003 ts_tablet_manager.cc:350] Time spent starting tablet 6cc27b921ae14cb4ae55c0401e4c58e9: real 0.001s user 0.001s sys 0.000s I1102 12:55:09.424043 3108 tablet_bootstrap.cc:234] Tablet 4961d3da9c5d424f820aa4d6753fdeb6: No bootstrap required, opened a new log I1102 12:55:09.424062 3109 ts_tablet_manager.cc:350] Time spent starting tablet 6cc27b921ae14cb4ae55c0401e4c58e9: real 0.003s user 0.002s sys 0.001s I1102 12:55:09.424257 3106 raft_consensus.cc:284] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 50fad9a13654438a99c95c9d534c73f1 [FOLLOWER]: Becoming Follower/Learner I1102 12:55:09.424394 3106 ts_tablet_manager.cc:350] Time spent starting tablet 6cc27b921ae14cb4ae55c0401e4c58e9: real 0.001s user 0.001s sys 0.000s I1102 12:55:09.424473 3108 ts_tablet_manager.cc:330] Time spent tablet 4961d3da9c5d424f820aa4d6753fdeb6 bootstrap: real 0.010s user 0.006s sys 0.001s I1102 12:55:09.425355 3108 raft_consensus.cc:227] T 4961d3da9c5d424f820aa4d6753fdeb6 P 50fad9a13654438a99c95c9d534c73f1 [LEADER]: Becoming Leader I1102 12:55:09.426620 3108 ts_tablet_manager.cc:350] Time spent starting tablet 4961d3da9c5d424f820aa4d6753fdeb6: real 0.002s user 0.002s sys 0.000s W1102 12:55:09.463419 3124 raft_consensus_state.cc:462] T 6cc27b921ae14cb4ae55c0401e4c58e9 P eedd3ed575384fcf880ceb61df2e3cb6 [LEADER]: Can't advance the committed index across term boundaries yet. Last committed operation was: term: 0 index: 0 New majority replicated is: term: 0 index: 0. Current term is: 1 W1102 12:55:09.464095 3113 raft_consensus_state.cc:462] T 6cc27b921ae14cb4ae55c0401e4c58e9 P eedd3ed575384fcf880ceb61df2e3cb6 [LEADER]: Can't advance the committed index across term boundaries yet. Last committed operation was: term: 0 index: 0 New majority replicated is: term: 0 index: 0. Current term is: 1 I1102 12:55:09.464825 3089 raft_consensus.cc:582] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 50fad9a13654438a99c95c9d534c73f1 [FOLLOWER]: Refusing update from remote peer: Log matching property violated. Last replicated by replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. I1102 12:55:09.464829 2982 raft_consensus.cc:582] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: Refusing update from remote peer: Log matching property violated. Last replicated by replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. I1102 12:55:09.465072 3017 consensus_queue.cc:453] Connected to peer: 530d10cfdcdf43d891aca53a64d58a88 whose last received watermark is: term: 0 index: 0 I1102 12:55:09.465165 3018 consensus_queue.cc:453] Connected to peer: 50fad9a13654438a99c95c9d534c73f1 whose last received watermark is: term: 0 index: 0 I1102 12:55:09.469925 3113 tablet_peer.cc:213] Configuration changed for tablet: 6cc27b921ae14cb4ae55c0401e4c58e9 in peer: eedd3ed575384fcf880ceb61df2e3cb6 Changed from: seqno: -1 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: LEADER last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } W1102 12:55:09.470809 3131 raft_consensus_state.cc:462] T 4961d3da9c5d424f820aa4d6753fdeb6 P 50fad9a13654438a99c95c9d534c73f1 [LEADER]: Can't advance the committed index across term boundaries yet. Last committed operation was: term: 0 index: 0 New majority replicated is: term: 0 index: 0. Current term is: 1 I1102 12:55:09.471374 3017 raft_consensus_state.cc:455] T 6cc27b921ae14cb4ae55c0401e4c58e9 P eedd3ed575384fcf880ceb61df2e3cb6 [LEADER]: Advanced the committed_index across terms. Last committed operation was: term: 0 index: 0 New committed index is: term: 1 index: 2 W1102 12:55:09.471413 3125 raft_consensus_state.cc:462] T 4961d3da9c5d424f820aa4d6753fdeb6 P 50fad9a13654438a99c95c9d534c73f1 [LEADER]: Can't advance the committed index across term boundaries yet. Last committed operation was: term: 0 index: 0 New majority replicated is: term: 0 index: 0. Current term is: 1 I1102 12:55:09.473642 2984 raft_consensus.cc:582] T 4961d3da9c5d424f820aa4d6753fdeb6 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: Refusing update from remote peer: Log matching property violated. Last replicated by replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. I1102 12:55:09.473759 3044 raft_consensus.cc:582] T 4961d3da9c5d424f820aa4d6753fdeb6 P eedd3ed575384fcf880ceb61df2e3cb6 [FOLLOWER]: Refusing update from remote peer: Log matching property violated. Last replicated by replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. I1102 12:55:09.473956 3060 consensus_queue.cc:453] Connected to peer: 530d10cfdcdf43d891aca53a64d58a88 whose last received watermark is: term: 0 index: 0 I1102 12:55:09.474071 3062 consensus_queue.cc:453] Connected to peer: eedd3ed575384fcf880ceb61df2e3cb6 whose last received watermark is: term: 0 index: 0 I1102 12:55:09.477654 3062 raft_consensus_state.cc:455] T 4961d3da9c5d424f820aa4d6753fdeb6 P 50fad9a13654438a99c95c9d534c73f1 [LEADER]: Advanced the committed_index across terms. Last committed operation was: term: 0 index: 0 New committed index is: term: 1 index: 2 I1102 12:55:09.477710 3125 tablet_peer.cc:213] Configuration changed for tablet: 4961d3da9c5d424f820aa4d6753fdeb6 in peer: 50fad9a13654438a99c95c9d534c73f1 Changed from: seqno: -1 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: LEADER last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } I1102 12:55:09.478222 2929 catalog_manager.cc:1127] Tablet: 6cc27b921ae14cb4ae55c0401e4c58e9 reported quorum change. New quorum: seqno: 0 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: LEADER last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } I1102 12:55:09.481031 2924 catalog_manager.cc:1127] Tablet: 4961d3da9c5d424f820aa4d6753fdeb6 reported quorum change. New quorum: seqno: 0 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: LEADER last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } I1102 12:55:09.968325 3121 tablet_peer.cc:213] Configuration changed for tablet: 6cc27b921ae14cb4ae55c0401e4c58e9 in peer: 50fad9a13654438a99c95c9d534c73f1 Changed from: seqno: -1 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: LEADER last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } I1102 12:55:09.968531 3116 tablet_peer.cc:213] Configuration changed for tablet: 6cc27b921ae14cb4ae55c0401e4c58e9 in peer: 530d10cfdcdf43d891aca53a64d58a88 Changed from: seqno: -1 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: LEADER last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } I1102 12:55:09.980470 3117 tablet_peer.cc:213] Configuration changed for tablet: 4961d3da9c5d424f820aa4d6753fdeb6 in peer: eedd3ed575384fcf880ceb61df2e3cb6 Changed from: seqno: -1 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: LEADER last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } I1102 12:55:09.981575 3112 tablet_peer.cc:213] Configuration changed for tablet: 4961d3da9c5d424f820aa4d6753fdeb6 in peer: 530d10cfdcdf43d891aca53a64d58a88 Changed from: seqno: -1 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "eedd3ed575384fcf880ceb61df2e3cb6" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42865 } } peers { permanent_uuid: "50fad9a13654438a99c95c9d534c73f1" role: LEADER last_known_addr { host: "127.0.0.1" port: 45677 } } peers { permanent_uuid: "530d10cfdcdf43d891aca53a64d58a88" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 49414 } } I1102 12:55:10.984280 31819 client-test.cc:1480] Killing TS running on 127.0.0.1:42865 I1102 12:55:10.984419 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:11.004029 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:11.004083 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:11.004128 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 6cc27b921ae14cb4ae55c0401e4c58e9 I1102 12:55:11.004159 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(6cc27b921ae14cb4ae55c0401e4c58e9) I1102 12:55:11.004178 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(6cc27b921ae14cb4ae55c0401e4c58e9) I1102 12:55:11.004194 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(6cc27b921ae14cb4ae55c0401e4c58e9) I1102 12:55:11.004226 31819 raft_consensus.cc:769] T 6cc27b921ae14cb4ae55c0401e4c58e9 P eedd3ed575384fcf880ceb61df2e3cb6 [LEADER]: Raft consensus shutting down. I1102 12:55:11.004407 31819 consensus_peers.cc:387] Closing peer: 50fad9a13654438a99c95c9d534c73f1 I1102 12:55:11.004587 31819 consensus_peers.cc:387] Closing peer: 530d10cfdcdf43d891aca53a64d58a88 I1102 12:55:11.004740 31819 consensus_peers.cc:387] Closing peer: eedd3ed575384fcf880ceb61df2e3cb6 I1102 12:55:11.004781 31819 raft_consensus_state.cc:377] T 6cc27b921ae14cb4ae55c0401e4c58e9 P eedd3ed575384fcf880ceb61df2e3cb6 [LEADER]: Trying to abort 0 pending transactions. I1102 12:55:11.004812 31819 raft_consensus_state.cc:404] T 6cc27b921ae14cb4ae55c0401e4c58e9 P eedd3ed575384fcf880ceb61df2e3cb6 [LEADER]: Waiting on 0 outstanding applies: I1102 12:55:11.004832 31819 raft_consensus_state.cc:408] T 6cc27b921ae14cb4ae55c0401e4c58e9 P eedd3ed575384fcf880ceb61df2e3cb6 [LEADER]: All local commits completed. I1102 12:55:11.004859 31819 raft_consensus.cc:789] T 6cc27b921ae14cb4ae55c0401e4c58e9 P eedd3ed575384fcf880ceb61df2e3cb6 [LEADER]: Raft consensus Shutdown! I1102 12:55:11.005422 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 4961d3da9c5d424f820aa4d6753fdeb6 I1102 12:55:11.005458 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(4961d3da9c5d424f820aa4d6753fdeb6) I1102 12:55:11.005475 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(4961d3da9c5d424f820aa4d6753fdeb6) I1102 12:55:11.005489 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(4961d3da9c5d424f820aa4d6753fdeb6) I1102 12:55:11.005512 31819 raft_consensus.cc:769] T 4961d3da9c5d424f820aa4d6753fdeb6 P eedd3ed575384fcf880ceb61df2e3cb6 [FOLLOWER]: Raft consensus shutting down. I1102 12:55:11.005550 31819 raft_consensus_state.cc:377] T 4961d3da9c5d424f820aa4d6753fdeb6 P eedd3ed575384fcf880ceb61df2e3cb6 [FOLLOWER]: Trying to abort 0 pending transactions. I1102 12:55:11.005573 31819 raft_consensus_state.cc:404] T 4961d3da9c5d424f820aa4d6753fdeb6 P eedd3ed575384fcf880ceb61df2e3cb6 [FOLLOWER]: Waiting on 0 outstanding applies: I1102 12:55:11.005591 31819 raft_consensus_state.cc:408] T 4961d3da9c5d424f820aa4d6753fdeb6 P eedd3ed575384fcf880ceb61df2e3cb6 [FOLLOWER]: All local commits completed. I1102 12:55:11.005610 31819 raft_consensus.cc:789] T 4961d3da9c5d424f820aa4d6753fdeb6 P eedd3ed575384fcf880ceb61df2e3cb6 [FOLLOWER]: Raft consensus Shutdown! I1102 12:55:11.010195 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:11.010221 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:11.010260 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! W1102 12:55:11.012101 3156 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:42865: connect: Connection refused (error 111) W1102 12:55:11.012223 31819 meta_cache.cc:159] Replica 6cc27b921ae14cb4ae55c0401e4c58e9 on ts eedd3ed575384fcf880ceb61df2e3cb6 has failed: Network error: TS failed: client connection negotiation failed: client Connection to 127.0.0.1:42865: connect: Connection refused (error 111) W1102 12:55:11.012265 31819 meta_cache.cc:159] Replica 4961d3da9c5d424f820aa4d6753fdeb6 on ts eedd3ed575384fcf880ceb61df2e3cb6 has failed: Network error: TS failed: client connection negotiation failed: client Connection to 127.0.0.1:42865: connect: Connection refused (error 111) W1102 12:55:11.019122 3156 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:42865: connect: Connection refused (error 111) W1102 12:55:11.019438 31819 meta_cache.cc:159] Replica 6cc27b921ae14cb4ae55c0401e4c58e9 on ts eedd3ed575384fcf880ceb61df2e3cb6 has failed: Network error: TS failed: client connection negotiation failed: client Connection to 127.0.0.1:42865: connect: Connection refused (error 111) W1102 12:55:11.019556 31819 meta_cache.cc:159] Replica 4961d3da9c5d424f820aa4d6753fdeb6 on ts eedd3ed575384fcf880ceb61df2e3cb6 has failed: Network error: TS failed: client connection negotiation failed: client Connection to 127.0.0.1:42865: connect: Connection refused (error 111) I1102 12:55:11.024816 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:11.043617 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:11.043715 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:11.043797 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 150dc7143d7b4b539ab695ff7e1757b9 I1102 12:55:11.043853 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(150dc7143d7b4b539ab695ff7e1757b9) I1102 12:55:11.043889 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(150dc7143d7b4b539ab695ff7e1757b9) I1102 12:55:11.043920 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(150dc7143d7b4b539ab695ff7e1757b9) I1102 12:55:11.044862 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 4b8d89d654e04b96b5955951098ddfce I1102 12:55:11.044927 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(4b8d89d654e04b96b5955951098ddfce) I1102 12:55:11.044961 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(4b8d89d654e04b96b5955951098ddfce) I1102 12:55:11.044991 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(4b8d89d654e04b96b5955951098ddfce) I1102 12:55:11.045891 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 6cc27b921ae14cb4ae55c0401e4c58e9 I1102 12:55:11.045953 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(6cc27b921ae14cb4ae55c0401e4c58e9) I1102 12:55:11.045987 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(6cc27b921ae14cb4ae55c0401e4c58e9) I1102 12:55:11.046015 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(6cc27b921ae14cb4ae55c0401e4c58e9) I1102 12:55:11.046064 31819 raft_consensus.cc:769] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: Raft consensus shutting down. I1102 12:55:11.046154 31819 raft_consensus_state.cc:377] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: Trying to abort 0 pending transactions. I1102 12:55:11.046200 31819 raft_consensus_state.cc:404] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: Waiting on 0 outstanding applies: I1102 12:55:11.046241 31819 raft_consensus_state.cc:408] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: All local commits completed. I1102 12:55:11.046279 31819 raft_consensus.cc:789] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: Raft consensus Shutdown! I1102 12:55:11.047111 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 4961d3da9c5d424f820aa4d6753fdeb6 I1102 12:55:11.047171 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(4961d3da9c5d424f820aa4d6753fdeb6) I1102 12:55:11.047205 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(4961d3da9c5d424f820aa4d6753fdeb6) I1102 12:55:11.047231 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(4961d3da9c5d424f820aa4d6753fdeb6) I1102 12:55:11.047276 31819 raft_consensus.cc:769] T 4961d3da9c5d424f820aa4d6753fdeb6 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: Raft consensus shutting down. I1102 12:55:11.047343 31819 raft_consensus_state.cc:377] T 4961d3da9c5d424f820aa4d6753fdeb6 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: Trying to abort 0 pending transactions. I1102 12:55:11.047384 31819 raft_consensus_state.cc:404] T 4961d3da9c5d424f820aa4d6753fdeb6 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: Waiting on 0 outstanding applies: I1102 12:55:11.047422 31819 raft_consensus_state.cc:408] T 4961d3da9c5d424f820aa4d6753fdeb6 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: All local commits completed. I1102 12:55:11.047458 31819 raft_consensus.cc:789] T 4961d3da9c5d424f820aa4d6753fdeb6 P 530d10cfdcdf43d891aca53a64d58a88 [FOLLOWER]: Raft consensus Shutdown! I1102 12:55:11.048187 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 7deb92afd493467fb4355ff7aa9c3b27 I1102 12:55:11.048245 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(7deb92afd493467fb4355ff7aa9c3b27) I1102 12:55:11.048279 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(7deb92afd493467fb4355ff7aa9c3b27) I1102 12:55:11.048303 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(7deb92afd493467fb4355ff7aa9c3b27) I1102 12:55:11.061909 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:11.061939 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:11.061972 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:11.063026 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:11.076303 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:11.076354 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:11.076397 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 6cc27b921ae14cb4ae55c0401e4c58e9 I1102 12:55:11.076431 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(6cc27b921ae14cb4ae55c0401e4c58e9) I1102 12:55:11.076453 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(6cc27b921ae14cb4ae55c0401e4c58e9) I1102 12:55:11.076472 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(6cc27b921ae14cb4ae55c0401e4c58e9) I1102 12:55:11.076504 31819 raft_consensus.cc:769] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 50fad9a13654438a99c95c9d534c73f1 [FOLLOWER]: Raft consensus shutting down. I1102 12:55:11.076557 31819 raft_consensus_state.cc:377] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 50fad9a13654438a99c95c9d534c73f1 [FOLLOWER]: Trying to abort 0 pending transactions. I1102 12:55:11.076585 31819 raft_consensus_state.cc:404] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 50fad9a13654438a99c95c9d534c73f1 [FOLLOWER]: Waiting on 0 outstanding applies: I1102 12:55:11.076611 31819 raft_consensus_state.cc:408] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 50fad9a13654438a99c95c9d534c73f1 [FOLLOWER]: All local commits completed. I1102 12:55:11.076635 31819 raft_consensus.cc:789] T 6cc27b921ae14cb4ae55c0401e4c58e9 P 50fad9a13654438a99c95c9d534c73f1 [FOLLOWER]: Raft consensus Shutdown! I1102 12:55:11.077083 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 4961d3da9c5d424f820aa4d6753fdeb6 I1102 12:55:11.077132 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(4961d3da9c5d424f820aa4d6753fdeb6) I1102 12:55:11.077155 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(4961d3da9c5d424f820aa4d6753fdeb6) I1102 12:55:11.077174 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(4961d3da9c5d424f820aa4d6753fdeb6) I1102 12:55:11.077201 31819 raft_consensus.cc:769] T 4961d3da9c5d424f820aa4d6753fdeb6 P 50fad9a13654438a99c95c9d534c73f1 [LEADER]: Raft consensus shutting down. I1102 12:55:11.077333 31819 consensus_peers.cc:387] Closing peer: 50fad9a13654438a99c95c9d534c73f1 I1102 12:55:11.077460 31819 consensus_peers.cc:387] Closing peer: 530d10cfdcdf43d891aca53a64d58a88 I1102 12:55:11.077576 31819 consensus_peers.cc:387] Closing peer: eedd3ed575384fcf880ceb61df2e3cb6 I1102 12:55:11.077610 31819 raft_consensus_state.cc:377] T 4961d3da9c5d424f820aa4d6753fdeb6 P 50fad9a13654438a99c95c9d534c73f1 [LEADER]: Trying to abort 0 pending transactions. I1102 12:55:11.077636 31819 raft_consensus_state.cc:404] T 4961d3da9c5d424f820aa4d6753fdeb6 P 50fad9a13654438a99c95c9d534c73f1 [LEADER]: Waiting on 0 outstanding applies: I1102 12:55:11.077661 31819 raft_consensus_state.cc:408] T 4961d3da9c5d424f820aa4d6753fdeb6 P 50fad9a13654438a99c95c9d534c73f1 [LEADER]: All local commits completed. I1102 12:55:11.077684 31819 raft_consensus.cc:789] T 4961d3da9c5d424f820aa4d6753fdeb6 P 50fad9a13654438a99c95c9d534c73f1 [LEADER]: Raft consensus Shutdown! I1102 12:55:11.081058 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:11.081079 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:11.081127 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:11.082051 31819 master.cc:124] Master@127.0.0.1:40072 shutting down... I1102 12:55:11.094179 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:11.094606 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:11.094933 31819 master.cc:127] Master@127.0.0.1:40072 shutdown complete. [ OK ] ClientTest.TestReplicatedMultiTabletTableFailover (2155 ms) [ RUN ] ClientTest.TestReplicatedTabletWritesWithLeaderElection I1102 12:55:11.384852 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedTabletWritesWithLeaderElection.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "daa2b08d0c18446a8561096c8258857e" format_stamp: "Formatted at 2014-11-02 20:55:11 on qiwi" I1102 12:55:11.385020 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedTabletWritesWithLeaderElection.1414961696528764-31819/minicluster-data/master-0-root uuid: "daa2b08d0c18446a8561096c8258857e" format_stamp: "Formatted at 2014-11-02 20:55:11 on qiwi" I1102 12:55:11.386111 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 498778 I1102 12:55:11.390458 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:11.390481 31819 webserver.cc:117] Document root disabled I1102 12:55:11.390774 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:42389 I1102 12:55:11.392601 3183 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:11.395558 3183 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:11.396733 3183 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:11.398704 3185 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: daa2b08d0c18446a8561096c8258857e Changed from: seqno: 0 local: true peers { permanent_uuid: "daa2b08d0c18446a8561096c8258857e" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "daa2b08d0c18446a8561096c8258857e" role: LEADER } I1102 12:55:11.398782 3185 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "daa2b08d0c18446a8561096c8258857e" role: LEADER } I1102 12:55:11.399544 3183 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:11.402487 3183 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:11.403576 3183 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:11.405033 3183 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:11.405527 3189 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: daa2b08d0c18446a8561096c8258857e Changed from: seqno: 0 local: true peers { permanent_uuid: "daa2b08d0c18446a8561096c8258857e" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "daa2b08d0c18446a8561096c8258857e" role: LEADER } I1102 12:55:11.405591 3183 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:11.405594 3189 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "daa2b08d0c18446a8561096c8258857e" role: LEADER } I1102 12:55:11.422670 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedTabletWritesWithLeaderElection.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "169c4fbcaa1046a3884fa2baafedfc89" format_stamp: "Formatted at 2014-11-02 20:55:11 on qiwi" I1102 12:55:11.422770 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedTabletWritesWithLeaderElection.1414961696528764-31819/minicluster-data/ts-0-root uuid: "169c4fbcaa1046a3884fa2baafedfc89" format_stamp: "Formatted at 2014-11-02 20:55:11 on qiwi" I1102 12:55:11.423635 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 498778 I1102 12:55:11.427884 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:11.427902 31819 webserver.cc:117] Document root disabled I1102 12:55:11.428128 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:34538 I1102 12:55:11.428256 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:11.429816 3232 heartbeater.cc:227] Registering TS with master... I1102 12:55:11.429873 3232 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:11.430220 3162 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" instance_seqno: 1414961711427618 } with Master I1102 12:55:11.430573 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00219728s I1102 12:55:11.433763 3164 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:53725: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:11.439884 3210 tablet_service.cc:264] Processing CreateTablet for tablet 465bd42c612e45c7a34fc5eacd8300d3 (table=client-testtb [id=ae33fd7b39ad4022b046a4b61ce6e7dc]), range=["", "\x00\x00\x00\t"] I1102 12:55:11.439946 3211 tablet_service.cc:264] Processing CreateTablet for tablet 171f23b4a9974ca4ace7e6df2336809f (table=client-testtb [id=ae33fd7b39ad4022b046a4b61ce6e7dc]), range=["\x00\x00\x00\t", ""] I1102 12:55:11.440943 3211 ts_tablet_manager.cc:460] Registered tablet 171f23b4a9974ca4ace7e6df2336809f I1102 12:55:11.440971 3210 ts_tablet_manager.cc:460] Registered tablet 465bd42c612e45c7a34fc5eacd8300d3 I1102 12:55:11.441412 3241 ts_tablet_manager.cc:325] Bootstrapping tablet: 171f23b4a9974ca4ace7e6df2336809f I1102 12:55:11.441431 3242 ts_tablet_manager.cc:325] Bootstrapping tablet: 465bd42c612e45c7a34fc5eacd8300d3 I1102 12:55:11.441913 3241 tablet_bootstrap.cc:234] Tablet 171f23b4a9974ca4ace7e6df2336809f: Bootstrap starting. I1102 12:55:11.441929 3242 tablet_bootstrap.cc:234] Tablet 465bd42c612e45c7a34fc5eacd8300d3: Bootstrap starting. I1102 12:55:11.444497 3241 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 171f23b4a9974ca4ace7e6df2336809f creating new one. I1102 12:55:11.444512 3242 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 465bd42c612e45c7a34fc5eacd8300d3 creating new one. I1102 12:55:11.445801 3241 tablet_bootstrap.cc:234] Tablet 171f23b4a9974ca4ace7e6df2336809f: No bootstrap required, opened a new log I1102 12:55:11.445804 3242 tablet_bootstrap.cc:234] Tablet 465bd42c612e45c7a34fc5eacd8300d3: No bootstrap required, opened a new log I1102 12:55:11.446030 3241 ts_tablet_manager.cc:330] Time spent tablet 171f23b4a9974ca4ace7e6df2336809f bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:11.446030 3242 ts_tablet_manager.cc:330] Time spent tablet 465bd42c612e45c7a34fc5eacd8300d3 bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:11.447471 3242 ts_tablet_manager.cc:350] Time spent starting tablet 465bd42c612e45c7a34fc5eacd8300d3: real 0.001s user 0.000s sys 0.000s I1102 12:55:11.447604 3241 ts_tablet_manager.cc:350] Time spent starting tablet 171f23b4a9974ca4ace7e6df2336809f: real 0.002s user 0.001s sys 0.000s I1102 12:55:11.448108 3245 tablet_peer.cc:213] Configuration changed for tablet: 171f23b4a9974ca4ace7e6df2336809f in peer: 169c4fbcaa1046a3884fa2baafedfc89 Changed from: seqno: -1 local: true peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 41926 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: LEADER last_known_addr { host: "127.0.0.1" port: 41926 } } I1102 12:55:11.448101 3246 tablet_peer.cc:213] Configuration changed for tablet: 465bd42c612e45c7a34fc5eacd8300d3 in peer: 169c4fbcaa1046a3884fa2baafedfc89 Changed from: seqno: -1 local: true peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 41926 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: LEADER last_known_addr { host: "127.0.0.1" port: 41926 } } I1102 12:55:11.451869 3162 catalog_manager.cc:1127] Tablet: 465bd42c612e45c7a34fc5eacd8300d3 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: LEADER last_known_addr { host: "127.0.0.1" port: 41926 } } I1102 12:55:11.453439 3162 catalog_manager.cc:1127] Tablet: 171f23b4a9974ca4ace7e6df2336809f reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: LEADER last_known_addr { host: "127.0.0.1" port: 41926 } } I1102 12:55:11.457552 3166 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:53725: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:11.462937 3212 tablet_service.cc:264] Processing CreateTablet for tablet c34828aa0ed04caabddafeebfa0d11a9 (table=client-testtb2 [id=99ec2ad159a145cb8f7046d545a1c150]), range=["", ""] I1102 12:55:11.464459 3212 ts_tablet_manager.cc:460] Registered tablet c34828aa0ed04caabddafeebfa0d11a9 I1102 12:55:11.464603 3242 ts_tablet_manager.cc:325] Bootstrapping tablet: c34828aa0ed04caabddafeebfa0d11a9 I1102 12:55:11.465512 3242 tablet_bootstrap.cc:234] Tablet c34828aa0ed04caabddafeebfa0d11a9: Bootstrap starting. I1102 12:55:11.469707 3242 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: c34828aa0ed04caabddafeebfa0d11a9 creating new one. I1102 12:55:11.470630 3242 tablet_bootstrap.cc:234] Tablet c34828aa0ed04caabddafeebfa0d11a9: No bootstrap required, opened a new log I1102 12:55:11.470811 3242 ts_tablet_manager.cc:330] Time spent tablet c34828aa0ed04caabddafeebfa0d11a9 bootstrap: real 0.006s user 0.005s sys 0.001s I1102 12:55:11.471741 3242 ts_tablet_manager.cc:350] Time spent starting tablet c34828aa0ed04caabddafeebfa0d11a9: real 0.001s user 0.000s sys 0.000s I1102 12:55:11.472023 3252 tablet_peer.cc:213] Configuration changed for tablet: c34828aa0ed04caabddafeebfa0d11a9 in peer: 169c4fbcaa1046a3884fa2baafedfc89 Changed from: seqno: -1 local: true peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 41926 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: LEADER last_known_addr { host: "127.0.0.1" port: 41926 } } I1102 12:55:11.477027 3164 catalog_manager.cc:1127] Tablet: c34828aa0ed04caabddafeebfa0d11a9 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: LEADER last_known_addr { host: "127.0.0.1" port: 41926 } } I1102 12:55:11.500413 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedTabletWritesWithLeaderElection.1414961696528764-31819/minicluster-data/ts-1-root/instance: uuid: "3fc07090a82c4b75abad25c747673e04" format_stamp: "Formatted at 2014-11-02 20:55:11 on qiwi" I1102 12:55:11.500512 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedTabletWritesWithLeaderElection.1414961696528764-31819/minicluster-data/ts-1-root uuid: "3fc07090a82c4b75abad25c747673e04" format_stamp: "Formatted at 2014-11-02 20:55:11 on qiwi" I1102 12:55:11.501302 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 498778 I1102 12:55:11.507446 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:11.507483 31819 webserver.cc:117] Document root disabled I1102 12:55:11.507977 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:58708 I1102 12:55:11.508270 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:11.511090 3296 heartbeater.cc:227] Registering TS with master... I1102 12:55:11.511204 3296 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:11.511852 3162 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" instance_seqno: 1414961711506863 } with Master I1102 12:55:11.527587 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedTabletWritesWithLeaderElection.1414961696528764-31819/minicluster-data/ts-2-root/instance: uuid: "575e6fa11bde4722a91fe1060f9c3283" format_stamp: "Formatted at 2014-11-02 20:55:11 on qiwi" I1102 12:55:11.527683 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestReplicatedTabletWritesWithLeaderElection.1414961696528764-31819/minicluster-data/ts-2-root uuid: "575e6fa11bde4722a91fe1060f9c3283" format_stamp: "Formatted at 2014-11-02 20:55:11 on qiwi" I1102 12:55:11.528527 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 498778 I1102 12:55:11.532393 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:11.532410 31819 webserver.cc:117] Document root disabled I1102 12:55:11.532657 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:60778 I1102 12:55:11.532783 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:11.535512 3338 heartbeater.cc:227] Registering TS with master... I1102 12:55:11.535617 3338 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:11.536371 3168 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" instance_seqno: 1414961711532131 } with Master I1102 12:55:11.536530 31819 mini_cluster.cc:268] 3 TS(s) registered with Master after 0.00349171s I1102 12:55:11.537679 3169 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:53725: name: "replicated_failover_on_writes" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 3 I1102 12:55:11.547513 3213 tablet_service.cc:264] Processing CreateTablet for tablet aa92c6fda6cd4409858dba0a0a063efc (table=replicated_failover_on_writes [id=5c89c578998d4efd8e67d40565d68bd7]), range=["", ""] I1102 12:55:11.549206 3316 tablet_service.cc:264] Processing CreateTablet for tablet aa92c6fda6cd4409858dba0a0a063efc (table=replicated_failover_on_writes [id=5c89c578998d4efd8e67d40565d68bd7]), range=["", ""] I1102 12:55:11.549576 3213 ts_tablet_manager.cc:460] Registered tablet aa92c6fda6cd4409858dba0a0a063efc I1102 12:55:11.549765 3241 ts_tablet_manager.cc:325] Bootstrapping tablet: aa92c6fda6cd4409858dba0a0a063efc I1102 12:55:11.549816 3271 tablet_service.cc:264] Processing CreateTablet for tablet aa92c6fda6cd4409858dba0a0a063efc (table=replicated_failover_on_writes [id=5c89c578998d4efd8e67d40565d68bd7]), range=["", ""] I1102 12:55:11.550798 3241 tablet_bootstrap.cc:234] Tablet aa92c6fda6cd4409858dba0a0a063efc: Bootstrap starting. I1102 12:55:11.551259 3316 ts_tablet_manager.cc:460] Registered tablet aa92c6fda6cd4409858dba0a0a063efc I1102 12:55:11.551640 3271 ts_tablet_manager.cc:460] Registered tablet aa92c6fda6cd4409858dba0a0a063efc I1102 12:55:11.551713 3342 ts_tablet_manager.cc:325] Bootstrapping tablet: aa92c6fda6cd4409858dba0a0a063efc I1102 12:55:11.552341 3343 ts_tablet_manager.cc:325] Bootstrapping tablet: aa92c6fda6cd4409858dba0a0a063efc I1102 12:55:11.552804 3342 tablet_bootstrap.cc:234] Tablet aa92c6fda6cd4409858dba0a0a063efc: Bootstrap starting. I1102 12:55:11.553270 3343 tablet_bootstrap.cc:234] Tablet aa92c6fda6cd4409858dba0a0a063efc: Bootstrap starting. I1102 12:55:11.554508 3241 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: aa92c6fda6cd4409858dba0a0a063efc creating new one. I1102 12:55:11.555367 3342 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: aa92c6fda6cd4409858dba0a0a063efc creating new one. I1102 12:55:11.555634 3343 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: aa92c6fda6cd4409858dba0a0a063efc creating new one. I1102 12:55:11.555858 3241 tablet_bootstrap.cc:234] Tablet aa92c6fda6cd4409858dba0a0a063efc: No bootstrap required, opened a new log I1102 12:55:11.556154 3241 ts_tablet_manager.cc:330] Time spent tablet aa92c6fda6cd4409858dba0a0a063efc bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:11.556387 3342 tablet_bootstrap.cc:234] Tablet aa92c6fda6cd4409858dba0a0a063efc: No bootstrap required, opened a new log I1102 12:55:11.556699 3343 tablet_bootstrap.cc:234] Tablet aa92c6fda6cd4409858dba0a0a063efc: No bootstrap required, opened a new log I1102 12:55:11.556725 3342 ts_tablet_manager.cc:330] Time spent tablet aa92c6fda6cd4409858dba0a0a063efc bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:11.556895 3241 raft_consensus.cc:284] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89 [FOLLOWER]: Becoming Follower/Learner I1102 12:55:11.557024 3241 ts_tablet_manager.cc:350] Time spent starting tablet aa92c6fda6cd4409858dba0a0a063efc: real 0.001s user 0.001s sys 0.000s I1102 12:55:11.557153 3343 ts_tablet_manager.cc:330] Time spent tablet aa92c6fda6cd4409858dba0a0a063efc bootstrap: real 0.005s user 0.004s sys 0.001s I1102 12:55:11.557919 3343 raft_consensus.cc:284] T aa92c6fda6cd4409858dba0a0a063efc P 3fc07090a82c4b75abad25c747673e04 [FOLLOWER]: Becoming Follower/Learner I1102 12:55:11.558043 3342 raft_consensus.cc:227] T aa92c6fda6cd4409858dba0a0a063efc P 575e6fa11bde4722a91fe1060f9c3283 [LEADER]: Becoming Leader I1102 12:55:11.558054 3343 ts_tablet_manager.cc:350] Time spent starting tablet aa92c6fda6cd4409858dba0a0a063efc: real 0.001s user 0.001s sys 0.000s I1102 12:55:11.559361 3342 ts_tablet_manager.cc:350] Time spent starting tablet aa92c6fda6cd4409858dba0a0a063efc: real 0.003s user 0.002s sys 0.001s W1102 12:55:11.610090 3353 raft_consensus_state.cc:462] T aa92c6fda6cd4409858dba0a0a063efc P 575e6fa11bde4722a91fe1060f9c3283 [LEADER]: Can't advance the committed index across term boundaries yet. Last committed operation was: term: 0 index: 0 New majority replicated is: term: 0 index: 0. Current term is: 1 W1102 12:55:11.610708 3348 raft_consensus_state.cc:462] T aa92c6fda6cd4409858dba0a0a063efc P 575e6fa11bde4722a91fe1060f9c3283 [LEADER]: Can't advance the committed index across term boundaries yet. Last committed operation was: term: 0 index: 0 New majority replicated is: term: 0 index: 0. Current term is: 1 I1102 12:55:11.611513 3220 raft_consensus.cc:582] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89 [FOLLOWER]: Refusing update from remote peer: Log matching property violated. Last replicated by replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. I1102 12:55:11.611543 3284 raft_consensus.cc:582] T aa92c6fda6cd4409858dba0a0a063efc P 3fc07090a82c4b75abad25c747673e04 [FOLLOWER]: Refusing update from remote peer: Log matching property violated. Last replicated by replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. I1102 12:55:11.611718 3301 consensus_queue.cc:453] Connected to peer: 3fc07090a82c4b75abad25c747673e04 whose last received watermark is: term: 0 index: 0 I1102 12:55:11.611747 3302 consensus_queue.cc:453] Connected to peer: 169c4fbcaa1046a3884fa2baafedfc89 whose last received watermark is: term: 0 index: 0 I1102 12:55:11.615234 3301 raft_consensus_state.cc:455] T aa92c6fda6cd4409858dba0a0a063efc P 575e6fa11bde4722a91fe1060f9c3283 [LEADER]: Advanced the committed_index across terms. Last committed operation was: term: 0 index: 0 New committed index is: term: 1 index: 2 I1102 12:55:11.615301 3348 tablet_peer.cc:213] Configuration changed for tablet: aa92c6fda6cd4409858dba0a0a063efc in peer: 575e6fa11bde4722a91fe1060f9c3283 Changed from: seqno: -1 local: false peers { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 44445 } } peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 41926 } } peers { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42531 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" role: LEADER last_known_addr { host: "127.0.0.1" port: 44445 } } peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 41926 } } peers { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42531 } } I1102 12:55:11.627794 3163 catalog_manager.cc:1127] Tablet: aa92c6fda6cd4409858dba0a0a063efc reported quorum change. New quorum: seqno: 0 local: false peers { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" role: LEADER last_known_addr { host: "127.0.0.1" port: 44445 } } peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 41926 } } peers { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42531 } } I1102 12:55:12.117687 3349 tablet_peer.cc:213] Configuration changed for tablet: aa92c6fda6cd4409858dba0a0a063efc in peer: 3fc07090a82c4b75abad25c747673e04 Changed from: seqno: -1 local: false peers { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 44445 } } peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 41926 } } peers { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42531 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" role: LEADER last_known_addr { host: "127.0.0.1" port: 44445 } } peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 41926 } } peers { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42531 } } I1102 12:55:12.118270 3345 tablet_peer.cc:213] Configuration changed for tablet: aa92c6fda6cd4409858dba0a0a063efc in peer: 169c4fbcaa1046a3884fa2baafedfc89 Changed from: seqno: -1 local: false peers { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 44445 } } peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 41926 } } peers { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42531 } } Changed to: seqno: 0 local: false peers { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" role: LEADER last_known_addr { host: "127.0.0.1" port: 44445 } } peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 41926 } } peers { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42531 } } I1102 12:55:13.134115 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:13.152914 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:13.153007 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:13.153081 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: aa92c6fda6cd4409858dba0a0a063efc I1102 12:55:13.153185 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(aa92c6fda6cd4409858dba0a0a063efc) I1102 12:55:13.153223 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(aa92c6fda6cd4409858dba0a0a063efc) I1102 12:55:13.153249 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(aa92c6fda6cd4409858dba0a0a063efc) I1102 12:55:13.153317 31819 raft_consensus.cc:769] T aa92c6fda6cd4409858dba0a0a063efc P 575e6fa11bde4722a91fe1060f9c3283 [LEADER]: Raft consensus shutting down. I1102 12:55:13.153720 31819 consensus_peers.cc:387] Closing peer: 169c4fbcaa1046a3884fa2baafedfc89 I1102 12:55:13.154109 31819 consensus_peers.cc:387] Closing peer: 3fc07090a82c4b75abad25c747673e04 I1102 12:55:13.154434 31819 consensus_peers.cc:387] Closing peer: 575e6fa11bde4722a91fe1060f9c3283 I1102 12:55:13.154547 31819 raft_consensus_state.cc:377] T aa92c6fda6cd4409858dba0a0a063efc P 575e6fa11bde4722a91fe1060f9c3283 [LEADER]: Trying to abort 0 pending transactions. I1102 12:55:13.154634 31819 raft_consensus_state.cc:404] T aa92c6fda6cd4409858dba0a0a063efc P 575e6fa11bde4722a91fe1060f9c3283 [LEADER]: Waiting on 0 outstanding applies: I1102 12:55:13.154708 31819 raft_consensus_state.cc:408] T aa92c6fda6cd4409858dba0a0a063efc P 575e6fa11bde4722a91fe1060f9c3283 [LEADER]: All local commits completed. I1102 12:55:13.154767 31819 raft_consensus.cc:789] T aa92c6fda6cd4409858dba0a0a063efc P 575e6fa11bde4722a91fe1060f9c3283 [LEADER]: Raft consensus Shutdown! I1102 12:55:13.159855 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:13.159900 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:13.159962 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! W1102 12:55:13.162271 3375 reactor.cc:199] Client call MakePeerLeader has no timeout set for connection id: {remote=127.0.0.1:41926, service_name=kudu.consensus.ConsensusService, user_credentials={real_user=alex, eff_user=}} I1102 12:55:13.165791 3225 raft_consensus.cc:172] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89 [CANDIDATE]: Starting election with quorum: seqno: 1 local: false peers { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 44445 } } peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 41926 } } peers { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42531 } } I1102 12:55:13.166378 3225 leader_election.cc:183] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89: Term 2 election: Requesting vote from peer 3fc07090a82c4b75abad25c747673e04 I1102 12:55:13.166481 3225 leader_election.cc:183] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89: Term 2 election: Requesting vote from peer 575e6fa11bde4722a91fe1060f9c3283 W1102 12:55:13.167042 3379 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111) W1102 12:55:13.167114 3194 leader_election.cc:218] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89: Term 2 election: RPC error from VoteRequest() call to peer 575e6fa11bde4722a91fe1060f9c3283: Network error: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111) I1102 12:55:13.168018 3289 raft_consensus.cc:933] Tablet aa92c6fda6cd4409858dba0a0a063efc: Replica 3fc07090a82c4b75abad25c747673e04: Leader election vote request: Granting yes vote for candidate 169c4fbcaa1046a3884fa2baafedfc89 in term 2. I1102 12:55:13.168213 3195 leader_election.cc:300] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89: Term 2 election: Vote granted by peer 3fc07090a82c4b75abad25c747673e04 I1102 12:55:13.168263 3195 leader_election.cc:273] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89: Term 2 election: Election decided. Result: candidate won. I1102 12:55:13.168306 3195 raft_consensus.cc:1009] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89 [CANDIDATE]: Leader election won for term 2 I1102 12:55:13.168510 3195 raft_consensus.cc:1026] Calling BecomeLeaderUnlocked()... I1102 12:55:13.168532 3195 raft_consensus.cc:227] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89 [LEADER]: Becoming Leader W1102 12:55:13.176215 3385 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111) W1102 12:55:13.176301 3236 meta_cache.cc:159] Replica aa92c6fda6cd4409858dba0a0a063efc on ts 575e6fa11bde4722a91fe1060f9c3283 has failed: Network error: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111) W1102 12:55:13.181618 3384 raft_consensus_state.cc:462] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89 [LEADER]: Can't advance the committed index across term boundaries yet. Last committed operation was: term: 1 index: 2 New majority replicated is: term: 1 index: 2. Current term is: 2 W1102 12:55:13.182131 3345 raft_consensus_state.cc:462] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89 [LEADER]: Can't advance the committed index across term boundaries yet. Last committed operation was: term: 1 index: 2 New majority replicated is: term: 0 index: 0. Current term is: 2 W1102 12:55:13.182317 3379 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111) I1102 12:55:13.182402 3290 raft_consensus.cc:582] T aa92c6fda6cd4409858dba0a0a063efc P 3fc07090a82c4b75abad25c747673e04 [FOLLOWER]: Refusing update from remote peer: Log matching property violated. Last replicated by replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 4. W1102 12:55:13.182407 3194 consensus_peers.cc:368] Couldn't send request to peer 575e6fa11bde4722a91fe1060f9c3283 for tablet aa92c6fda6cd4409858dba0a0a063efc ops [] Status: Network error: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 1 times. I1102 12:55:13.182646 3195 consensus_queue.cc:453] Connected to peer: 3fc07090a82c4b75abad25c747673e04 whose last received watermark is: term: 1 index: 2 I1102 12:55:13.185230 3195 raft_consensus_state.cc:455] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89 [LEADER]: Advanced the committed_index across terms. Last committed operation was: term: 1 index: 2 New committed index is: term: 2 index: 4 I1102 12:55:13.185379 3345 tablet_peer.cc:213] Configuration changed for tablet: aa92c6fda6cd4409858dba0a0a063efc in peer: 169c4fbcaa1046a3884fa2baafedfc89 Changed from: seqno: 0 local: false peers { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" role: LEADER last_known_addr { host: "127.0.0.1" port: 44445 } } peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 41926 } } peers { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42531 } } Changed to: seqno: 1 local: false peers { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 44445 } } peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: LEADER last_known_addr { host: "127.0.0.1" port: 41926 } } peers { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42531 } } I1102 12:55:13.187001 3169 catalog_manager.cc:1127] Tablet: aa92c6fda6cd4409858dba0a0a063efc reported quorum change. New quorum: seqno: 1 local: false peers { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 44445 } } peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: LEADER last_known_addr { host: "127.0.0.1" port: 41926 } } peers { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42531 } } W1102 12:55:13.670346 3379 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111) W1102 12:55:13.670598 3194 consensus_peers.cc:368] Couldn't send request to peer 575e6fa11bde4722a91fe1060f9c3283 for tablet aa92c6fda6cd4409858dba0a0a063efc ops [] Status: Network error: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 2 times. I1102 12:55:13.688730 3349 tablet_peer.cc:213] Configuration changed for tablet: aa92c6fda6cd4409858dba0a0a063efc in peer: 3fc07090a82c4b75abad25c747673e04 Changed from: seqno: 0 local: false peers { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" role: LEADER last_known_addr { host: "127.0.0.1" port: 44445 } } peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 41926 } } peers { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42531 } } Changed to: seqno: 1 local: false peers { permanent_uuid: "575e6fa11bde4722a91fe1060f9c3283" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 44445 } } peers { permanent_uuid: "169c4fbcaa1046a3884fa2baafedfc89" role: LEADER last_known_addr { host: "127.0.0.1" port: 41926 } } peers { permanent_uuid: "3fc07090a82c4b75abad25c747673e04" role: FOLLOWER last_known_addr { host: "127.0.0.1" port: 42531 } } W1102 12:55:14.169956 3379 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111) W1102 12:55:14.170037 3194 consensus_peers.cc:368] Couldn't send request to peer 575e6fa11bde4722a91fe1060f9c3283 for tablet aa92c6fda6cd4409858dba0a0a063efc ops [] Status: Network error: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 3 times. W1102 12:55:14.671356 3404 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111) W1102 12:55:14.671661 3194 consensus_peers.cc:368] Couldn't send request to peer 575e6fa11bde4722a91fe1060f9c3283 for tablet aa92c6fda6cd4409858dba0a0a063efc ops [] Status: Network error: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 4 times. W1102 12:55:14.698343 3405 negotiation.cc:195] Network error: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111) W1102 12:55:14.698613 31819 meta_cache.cc:159] Replica aa92c6fda6cd4409858dba0a0a063efc on ts 575e6fa11bde4722a91fe1060f9c3283 has failed: Network error: TS failed: client connection negotiation failed: client Connection to 127.0.0.1:44445: connect: Connection refused (error 111) I1102 12:55:14.704627 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:14.723594 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:14.723716 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:14.723819 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: aa92c6fda6cd4409858dba0a0a063efc I1102 12:55:14.723888 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(aa92c6fda6cd4409858dba0a0a063efc) I1102 12:55:14.723932 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(aa92c6fda6cd4409858dba0a0a063efc) I1102 12:55:14.723968 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(aa92c6fda6cd4409858dba0a0a063efc) I1102 12:55:14.724031 31819 raft_consensus.cc:769] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89 [LEADER]: Raft consensus shutting down. I1102 12:55:14.724479 31819 consensus_peers.cc:387] Closing peer: 169c4fbcaa1046a3884fa2baafedfc89 I1102 12:55:14.724905 31819 consensus_peers.cc:387] Closing peer: 3fc07090a82c4b75abad25c747673e04 I1102 12:55:14.725312 31819 consensus_peers.cc:387] Closing peer: 575e6fa11bde4722a91fe1060f9c3283 I1102 12:55:14.725532 31819 raft_consensus_state.cc:377] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89 [LEADER]: Trying to abort 0 pending transactions. I1102 12:55:14.725605 31819 raft_consensus_state.cc:404] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89 [LEADER]: Waiting on 0 outstanding applies: I1102 12:55:14.725656 31819 raft_consensus_state.cc:408] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89 [LEADER]: All local commits completed. I1102 12:55:14.725703 31819 raft_consensus.cc:789] T aa92c6fda6cd4409858dba0a0a063efc P 169c4fbcaa1046a3884fa2baafedfc89 [LEADER]: Raft consensus Shutdown! I1102 12:55:14.726843 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: c34828aa0ed04caabddafeebfa0d11a9 I1102 12:55:14.726927 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(c34828aa0ed04caabddafeebfa0d11a9) I1102 12:55:14.726972 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(c34828aa0ed04caabddafeebfa0d11a9) I1102 12:55:14.727010 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(c34828aa0ed04caabddafeebfa0d11a9) I1102 12:55:14.728004 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 465bd42c612e45c7a34fc5eacd8300d3 I1102 12:55:14.728080 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(465bd42c612e45c7a34fc5eacd8300d3) I1102 12:55:14.728122 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(465bd42c612e45c7a34fc5eacd8300d3) I1102 12:55:14.728152 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(465bd42c612e45c7a34fc5eacd8300d3) I1102 12:55:14.729126 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 171f23b4a9974ca4ace7e6df2336809f I1102 12:55:14.729229 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(171f23b4a9974ca4ace7e6df2336809f) I1102 12:55:14.729271 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(171f23b4a9974ca4ace7e6df2336809f) I1102 12:55:14.729303 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(171f23b4a9974ca4ace7e6df2336809f) I1102 12:55:14.740274 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:14.740296 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:14.740326 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:14.741078 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:14.759002 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:14.759115 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:14.759209 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: aa92c6fda6cd4409858dba0a0a063efc I1102 12:55:14.759281 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(aa92c6fda6cd4409858dba0a0a063efc) I1102 12:55:14.759332 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(aa92c6fda6cd4409858dba0a0a063efc) I1102 12:55:14.759377 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(aa92c6fda6cd4409858dba0a0a063efc) I1102 12:55:14.759448 31819 raft_consensus.cc:769] T aa92c6fda6cd4409858dba0a0a063efc P 3fc07090a82c4b75abad25c747673e04 [FOLLOWER]: Raft consensus shutting down. I1102 12:55:14.759562 31819 raft_consensus_state.cc:377] T aa92c6fda6cd4409858dba0a0a063efc P 3fc07090a82c4b75abad25c747673e04 [FOLLOWER]: Trying to abort 0 pending transactions. I1102 12:55:14.759624 31819 raft_consensus_state.cc:404] T aa92c6fda6cd4409858dba0a0a063efc P 3fc07090a82c4b75abad25c747673e04 [FOLLOWER]: Waiting on 0 outstanding applies: I1102 12:55:14.759683 31819 raft_consensus_state.cc:408] T aa92c6fda6cd4409858dba0a0a063efc P 3fc07090a82c4b75abad25c747673e04 [FOLLOWER]: All local commits completed. I1102 12:55:14.759742 31819 raft_consensus.cc:789] T aa92c6fda6cd4409858dba0a0a063efc P 3fc07090a82c4b75abad25c747673e04 [FOLLOWER]: Raft consensus Shutdown! I1102 12:55:14.763947 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:14.763990 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:14.764067 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:14.765825 31819 master.cc:124] Master@127.0.0.1:36726 shutting down... I1102 12:55:14.780172 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:14.780927 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:14.781663 31819 master.cc:127] Master@127.0.0.1:36726 shutdown complete. [ OK ] ClientTest.TestReplicatedTabletWritesWithLeaderElection (3695 ms) [ RUN ] ClientTest.TestRandomWriteOperation I1102 12:55:15.058604 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestRandomWriteOperation.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "842d5aef4ee54d79a9e1d7313e7cfe27" format_stamp: "Formatted at 2014-11-02 20:55:14 on qiwi" I1102 12:55:15.058972 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestRandomWriteOperation.1414961696528764-31819/minicluster-data/master-0-root uuid: "842d5aef4ee54d79a9e1d7313e7cfe27" format_stamp: "Formatted at 2014-11-02 20:55:14 on qiwi" I1102 12:55:15.061307 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 500778 I1102 12:55:15.070093 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:15.070117 31819 webserver.cc:117] Document root disabled I1102 12:55:15.070435 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:53447 I1102 12:55:15.072417 3432 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:15.075673 3432 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:15.076863 3432 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:15.079043 3434 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 842d5aef4ee54d79a9e1d7313e7cfe27 Changed from: seqno: 0 local: true peers { permanent_uuid: "842d5aef4ee54d79a9e1d7313e7cfe27" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "842d5aef4ee54d79a9e1d7313e7cfe27" role: LEADER } I1102 12:55:15.079131 3434 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "842d5aef4ee54d79a9e1d7313e7cfe27" role: LEADER } I1102 12:55:15.079829 3432 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:15.082340 3432 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:15.083184 3432 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:15.084213 3432 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:15.084584 3438 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 842d5aef4ee54d79a9e1d7313e7cfe27 Changed from: seqno: 0 local: true peers { permanent_uuid: "842d5aef4ee54d79a9e1d7313e7cfe27" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "842d5aef4ee54d79a9e1d7313e7cfe27" role: LEADER } I1102 12:55:15.084635 3432 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:15.084641 3438 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "842d5aef4ee54d79a9e1d7313e7cfe27" role: LEADER } I1102 12:55:15.099818 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestRandomWriteOperation.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "d2f0abb2a85a4faa9f31fa5540684fe8" format_stamp: "Formatted at 2014-11-02 20:55:15 on qiwi" I1102 12:55:15.100105 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestRandomWriteOperation.1414961696528764-31819/minicluster-data/ts-0-root uuid: "d2f0abb2a85a4faa9f31fa5540684fe8" format_stamp: "Formatted at 2014-11-02 20:55:15 on qiwi" I1102 12:55:15.102239 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 500778 I1102 12:55:15.108469 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:15.108487 31819 webserver.cc:117] Document root disabled I1102 12:55:15.108749 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:58255 I1102 12:55:15.108888 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:15.110592 3481 heartbeater.cc:227] Registering TS with master... I1102 12:55:15.110654 3481 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:15.110987 3411 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "d2f0abb2a85a4faa9f31fa5540684fe8" instance_seqno: 1414961715108169 } with Master I1102 12:55:15.111246 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00217146s I1102 12:55:15.114392 3413 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:50839: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:15.120600 3459 tablet_service.cc:264] Processing CreateTablet for tablet 830468cd583b418b82724d21527a66e4 (table=client-testtb [id=c7e9d9f9b4e14cbeb77952d86beaec33]), range=["", "\x00\x00\x00\t"] I1102 12:55:15.120650 3460 tablet_service.cc:264] Processing CreateTablet for tablet 29a87fd3d0b04b3bb4961ccd52b6625c (table=client-testtb [id=c7e9d9f9b4e14cbeb77952d86beaec33]), range=["\x00\x00\x00\t", ""] I1102 12:55:15.121645 3459 ts_tablet_manager.cc:460] Registered tablet 830468cd583b418b82724d21527a66e4 I1102 12:55:15.121671 3460 ts_tablet_manager.cc:460] Registered tablet 29a87fd3d0b04b3bb4961ccd52b6625c I1102 12:55:15.122129 3490 ts_tablet_manager.cc:325] Bootstrapping tablet: 830468cd583b418b82724d21527a66e4 I1102 12:55:15.122145 3491 ts_tablet_manager.cc:325] Bootstrapping tablet: 29a87fd3d0b04b3bb4961ccd52b6625c I1102 12:55:15.122603 3490 tablet_bootstrap.cc:234] Tablet 830468cd583b418b82724d21527a66e4: Bootstrap starting. I1102 12:55:15.122611 3491 tablet_bootstrap.cc:234] Tablet 29a87fd3d0b04b3bb4961ccd52b6625c: Bootstrap starting. I1102 12:55:15.125159 3491 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 29a87fd3d0b04b3bb4961ccd52b6625c creating new one. I1102 12:55:15.125356 3490 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 830468cd583b418b82724d21527a66e4 creating new one. I1102 12:55:15.126585 3491 tablet_bootstrap.cc:234] Tablet 29a87fd3d0b04b3bb4961ccd52b6625c: No bootstrap required, opened a new log I1102 12:55:15.126626 3490 tablet_bootstrap.cc:234] Tablet 830468cd583b418b82724d21527a66e4: No bootstrap required, opened a new log I1102 12:55:15.126907 3491 ts_tablet_manager.cc:330] Time spent tablet 29a87fd3d0b04b3bb4961ccd52b6625c bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:15.126940 3490 ts_tablet_manager.cc:330] Time spent tablet 830468cd583b418b82724d21527a66e4 bootstrap: real 0.005s user 0.003s sys 0.001s I1102 12:55:15.128506 3490 ts_tablet_manager.cc:350] Time spent starting tablet 830468cd583b418b82724d21527a66e4: real 0.002s user 0.001s sys 0.000s I1102 12:55:15.128707 3491 ts_tablet_manager.cc:350] Time spent starting tablet 29a87fd3d0b04b3bb4961ccd52b6625c: real 0.002s user 0.001s sys 0.000s I1102 12:55:15.129133 3495 tablet_peer.cc:213] Configuration changed for tablet: 830468cd583b418b82724d21527a66e4 in peer: d2f0abb2a85a4faa9f31fa5540684fe8 Changed from: seqno: -1 local: true peers { permanent_uuid: "d2f0abb2a85a4faa9f31fa5540684fe8" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 39018 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "d2f0abb2a85a4faa9f31fa5540684fe8" role: LEADER last_known_addr { host: "127.0.0.1" port: 39018 } } I1102 12:55:15.129281 3494 tablet_peer.cc:213] Configuration changed for tablet: 29a87fd3d0b04b3bb4961ccd52b6625c in peer: d2f0abb2a85a4faa9f31fa5540684fe8 Changed from: seqno: -1 local: true peers { permanent_uuid: "d2f0abb2a85a4faa9f31fa5540684fe8" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 39018 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "d2f0abb2a85a4faa9f31fa5540684fe8" role: LEADER last_known_addr { host: "127.0.0.1" port: 39018 } } I1102 12:55:15.132534 3411 catalog_manager.cc:1127] Tablet: 830468cd583b418b82724d21527a66e4 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "d2f0abb2a85a4faa9f31fa5540684fe8" role: LEADER last_known_addr { host: "127.0.0.1" port: 39018 } } I1102 12:55:15.134089 3411 catalog_manager.cc:1127] Tablet: 29a87fd3d0b04b3bb4961ccd52b6625c reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "d2f0abb2a85a4faa9f31fa5540684fe8" role: LEADER last_known_addr { host: "127.0.0.1" port: 39018 } } I1102 12:55:15.137771 3415 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:50839: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:15.142076 3461 tablet_service.cc:264] Processing CreateTablet for tablet c788f07d80d94bb183e65836a713f768 (table=client-testtb2 [id=e1a145e249464cb3be5d0b126adc2c13]), range=["", ""] I1102 12:55:15.143059 3461 ts_tablet_manager.cc:460] Registered tablet c788f07d80d94bb183e65836a713f768 I1102 12:55:15.143149 3490 ts_tablet_manager.cc:325] Bootstrapping tablet: c788f07d80d94bb183e65836a713f768 I1102 12:55:15.143559 3490 tablet_bootstrap.cc:234] Tablet c788f07d80d94bb183e65836a713f768: Bootstrap starting. I1102 12:55:15.146754 3490 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: c788f07d80d94bb183e65836a713f768 creating new one. I1102 12:55:15.148931 3490 tablet_bootstrap.cc:234] Tablet c788f07d80d94bb183e65836a713f768: No bootstrap required, opened a new log I1102 12:55:15.149463 3490 ts_tablet_manager.cc:330] Time spent tablet c788f07d80d94bb183e65836a713f768 bootstrap: real 0.006s user 0.004s sys 0.001s I1102 12:55:15.151864 3490 ts_tablet_manager.cc:350] Time spent starting tablet c788f07d80d94bb183e65836a713f768: real 0.002s user 0.001s sys 0.000s I1102 12:55:15.152462 3501 tablet_peer.cc:213] Configuration changed for tablet: c788f07d80d94bb183e65836a713f768 in peer: d2f0abb2a85a4faa9f31fa5540684fe8 Changed from: seqno: -1 local: true peers { permanent_uuid: "d2f0abb2a85a4faa9f31fa5540684fe8" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 39018 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "d2f0abb2a85a4faa9f31fa5540684fe8" role: LEADER last_known_addr { host: "127.0.0.1" port: 39018 } } I1102 12:55:15.158381 3413 catalog_manager.cc:1127] Tablet: c788f07d80d94bb183e65836a713f768 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "d2f0abb2a85a4faa9f31fa5540684fe8" role: LEADER last_known_addr { host: "127.0.0.1" port: 39018 } } I1102 12:55:15.210325 31819 client-test.cc:1665] Randomized mutations testing. I1102 12:55:15.210343 31819 test_util.cc:117] Using random seed: 1414961715 I1102 12:55:15.210355 31819 client-test.cc:1670] Correctness test 0 I1102 12:55:15.266773 31819 client-test.cc:1673] ...complete I1102 12:55:15.270566 31819 client-test.cc:1670] Correctness test 50 I1102 12:55:15.301501 31819 client-test.cc:1673] ...complete I1102 12:55:15.305143 31819 client-test.cc:1670] Correctness test 100 I1102 12:55:15.328354 31819 client-test.cc:1673] ...complete I1102 12:55:15.331845 31819 client-test.cc:1670] Correctness test 150 I1102 12:55:15.363371 31819 client-test.cc:1673] ...complete I1102 12:55:15.366834 31819 client-test.cc:1670] Correctness test 200 I1102 12:55:15.395262 31819 client-test.cc:1673] ...complete I1102 12:55:15.398821 31819 client-test.cc:1670] Correctness test 250 I1102 12:55:15.422924 31819 client-test.cc:1673] ...complete I1102 12:55:15.426481 31819 client-test.cc:1670] Correctness test 300 I1102 12:55:15.460451 31819 client-test.cc:1673] ...complete I1102 12:55:15.463955 31819 client-test.cc:1670] Correctness test 350 I1102 12:55:15.495957 31819 client-test.cc:1673] ...complete I1102 12:55:15.499552 31819 client-test.cc:1670] Correctness test 400 I1102 12:55:15.533453 31819 client-test.cc:1673] ...complete I1102 12:55:15.536906 31819 client-test.cc:1670] Correctness test 450 I1102 12:55:15.565107 31819 client-test.cc:1673] ...complete I1102 12:55:15.568581 31819 client-test.cc:1670] Correctness test 500 I1102 12:55:15.596228 31819 client-test.cc:1673] ...complete I1102 12:55:15.603148 31819 client-test.cc:1670] Correctness test 550 I1102 12:55:15.631431 31819 client-test.cc:1673] ...complete I1102 12:55:15.634960 31819 client-test.cc:1670] Correctness test 600 I1102 12:55:15.663641 31819 client-test.cc:1673] ...complete I1102 12:55:15.667130 31819 client-test.cc:1670] Correctness test 650 I1102 12:55:15.697280 31819 client-test.cc:1673] ...complete I1102 12:55:15.700765 31819 client-test.cc:1670] Correctness test 700 I1102 12:55:15.729301 31819 client-test.cc:1673] ...complete I1102 12:55:15.732884 31819 client-test.cc:1670] Correctness test 750 I1102 12:55:15.761839 31819 client-test.cc:1673] ...complete I1102 12:55:15.765316 31819 client-test.cc:1670] Correctness test 800 I1102 12:55:15.795640 31819 client-test.cc:1673] ...complete I1102 12:55:15.800098 31819 client-test.cc:1670] Correctness test 850 I1102 12:55:15.827581 31819 client-test.cc:1673] ...complete I1102 12:55:15.831042 31819 client-test.cc:1670] Correctness test 900 I1102 12:55:15.868615 31819 client-test.cc:1673] ...complete I1102 12:55:15.872082 31819 client-test.cc:1670] Correctness test 950 I1102 12:55:15.900274 31819 client-test.cc:1673] ...complete I1102 12:55:15.903854 31819 client-test.cc:1670] Correctness test 1000 I1102 12:55:15.932303 31819 client-test.cc:1673] ...complete I1102 12:55:15.957645 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:15.972759 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:15.972841 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:15.972909 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 830468cd583b418b82724d21527a66e4 I1102 12:55:15.972965 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(830468cd583b418b82724d21527a66e4) I1102 12:55:15.972995 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(830468cd583b418b82724d21527a66e4) I1102 12:55:15.973019 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(830468cd583b418b82724d21527a66e4) I1102 12:55:15.973968 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 29a87fd3d0b04b3bb4961ccd52b6625c I1102 12:55:15.974022 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(29a87fd3d0b04b3bb4961ccd52b6625c) I1102 12:55:15.974061 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(29a87fd3d0b04b3bb4961ccd52b6625c) I1102 12:55:15.974084 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(29a87fd3d0b04b3bb4961ccd52b6625c) I1102 12:55:15.974896 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: c788f07d80d94bb183e65836a713f768 I1102 12:55:15.974962 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(c788f07d80d94bb183e65836a713f768) I1102 12:55:15.974992 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(c788f07d80d94bb183e65836a713f768) I1102 12:55:15.975011 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(c788f07d80d94bb183e65836a713f768) I1102 12:55:15.988044 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:15.988068 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:15.988097 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:15.988932 31819 master.cc:124] Master@127.0.0.1:58206 shutting down... I1102 12:55:16.000747 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:16.001147 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:16.001410 31819 master.cc:127] Master@127.0.0.1:58206 shutdown complete. [ OK ] ClientTest.TestRandomWriteOperation (1210 ms) [ RUN ] ClientTest.TestMasterLookupPermits I1102 12:55:16.161178 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestMasterLookupPermits.1414961696528764-31819/minicluster-data/master-0-root/instance: uuid: "90a81ab58ef344d5b2ee7e86857620cc" format_stamp: "Formatted at 2014-11-02 20:55:16 on qiwi" I1102 12:55:16.161348 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestMasterLookupPermits.1414961696528764-31819/minicluster-data/master-0-root uuid: "90a81ab58ef344d5b2ee7e86857620cc" format_stamp: "Formatted at 2014-11-02 20:55:16 on qiwi" I1102 12:55:16.162328 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 501278 I1102 12:55:16.165340 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:16.165359 31819 webserver.cc:117] Document root disabled I1102 12:55:16.165591 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:41508 I1102 12:55:16.167050 3539 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: Bootstrap starting. I1102 12:55:16.169391 3539 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 11111111111111111111111111111111 creating new one. I1102 12:55:16.170367 3539 tablet_bootstrap.cc:234] Tablet 11111111111111111111111111111111: No bootstrap required, opened a new log I1102 12:55:16.172050 3541 tablet_peer.cc:213] Configuration changed for tablet: 11111111111111111111111111111111 in peer: 90a81ab58ef344d5b2ee7e86857620cc Changed from: seqno: 0 local: true peers { permanent_uuid: "90a81ab58ef344d5b2ee7e86857620cc" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "90a81ab58ef344d5b2ee7e86857620cc" role: LEADER } I1102 12:55:16.172116 3541 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "90a81ab58ef344d5b2ee7e86857620cc" role: LEADER } I1102 12:55:16.172590 3539 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: Bootstrap starting. I1102 12:55:16.174706 3539 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 00000000000000000000000000000000 creating new one. I1102 12:55:16.175624 3539 tablet_bootstrap.cc:234] Tablet 00000000000000000000000000000000: No bootstrap required, opened a new log I1102 12:55:16.176636 3539 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:16.177031 3545 tablet_peer.cc:213] Configuration changed for tablet: 00000000000000000000000000000000 in peer: 90a81ab58ef344d5b2ee7e86857620cc Changed from: seqno: 0 local: true peers { permanent_uuid: "90a81ab58ef344d5b2ee7e86857620cc" role: LEADER } Changed to: seqno: 1 local: true peers { permanent_uuid: "90a81ab58ef344d5b2ee7e86857620cc" role: LEADER } I1102 12:55:16.177089 3539 sys_tables.cc:281] SysTable tablet configured and running, proceeding with master startup. I1102 12:55:16.177088 3545 sys_tables.cc:217] SysTable state changed. New quorum config: seqno: 1 local: true peers { permanent_uuid: "90a81ab58ef344d5b2ee7e86857620cc" role: LEADER } I1102 12:55:16.189668 31819 fs_manager.cc:130] Generated new instance metadata in path /tmp/kudutest-1000/client-test.ClientTest.TestMasterLookupPermits.1414961696528764-31819/minicluster-data/ts-0-root/instance: uuid: "f82e6599031d4b1aa8525a5828820575" format_stamp: "Formatted at 2014-11-02 20:55:16 on qiwi" I1102 12:55:16.189769 31819 fs_manager.cc:82] Opened local filesystem: /tmp/kudutest-1000/client-test.ClientTest.TestMasterLookupPermits.1414961696528764-31819/minicluster-data/ts-0-root uuid: "f82e6599031d4b1aa8525a5828820575" format_stamp: "Formatted at 2014-11-02 20:55:16 on qiwi" I1102 12:55:16.190634 31819 hybrid_clock.cc:119] HybridClock initialized. Resolution in nanos?: 1 Wait times tolerance adjustment: 1.0005 Current error: 501278 I1102 12:55:16.194800 31819 webserver.cc:108] Starting webserver on 0.0.0.0:0 I1102 12:55:16.194816 31819 webserver.cc:117] Document root disabled I1102 12:55:16.195039 31819 webserver.cc:197] Webserver started. Bound to: 0.0.0.0:38634 I1102 12:55:16.195165 31819 maintenance_manager.cc:118] MaintenanceManager: targeting memory size of 25.106701 GB I1102 12:55:16.196562 3588 heartbeater.cc:227] Registering TS with master... I1102 12:55:16.196622 3588 heartbeater.cc:233] Sending a full tablet report to master... I1102 12:55:16.196941 3518 ts_manager.cc:66] Registered new tablet server { permanent_uuid: "f82e6599031d4b1aa8525a5828820575" instance_seqno: 1414961716194545 } with Master I1102 12:55:16.197527 31819 mini_cluster.cc:268] 1 TS(s) registered with Master after 0.00218642s I1102 12:55:16.200145 3520 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:37162: name: "client-testtb" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } pre_split_keys: "\000\000\000\t" num_replicas: 1 I1102 12:55:16.205235 3566 tablet_service.cc:264] Processing CreateTablet for tablet 4cad4a9ff0384a9aa991e073a8c3290f (table=client-testtb [id=fa6c5a15a1284adda30bbcc5ddbd3118]), range=["", "\x00\x00\x00\t"] I1102 12:55:16.205283 3567 tablet_service.cc:264] Processing CreateTablet for tablet abb97e69180c4801a70e73034ac53fc6 (table=client-testtb [id=fa6c5a15a1284adda30bbcc5ddbd3118]), range=["\x00\x00\x00\t", ""] I1102 12:55:16.206140 3567 ts_tablet_manager.cc:460] Registered tablet abb97e69180c4801a70e73034ac53fc6 I1102 12:55:16.206209 3566 ts_tablet_manager.cc:460] Registered tablet 4cad4a9ff0384a9aa991e073a8c3290f I1102 12:55:16.206498 3597 ts_tablet_manager.cc:325] Bootstrapping tablet: abb97e69180c4801a70e73034ac53fc6 I1102 12:55:16.206498 3598 ts_tablet_manager.cc:325] Bootstrapping tablet: 4cad4a9ff0384a9aa991e073a8c3290f I1102 12:55:16.207097 3598 tablet_bootstrap.cc:234] Tablet 4cad4a9ff0384a9aa991e073a8c3290f: Bootstrap starting. I1102 12:55:16.207098 3597 tablet_bootstrap.cc:234] Tablet abb97e69180c4801a70e73034ac53fc6: Bootstrap starting. I1102 12:55:16.210366 3598 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 4cad4a9ff0384a9aa991e073a8c3290f creating new one. I1102 12:55:16.210377 3597 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: abb97e69180c4801a70e73034ac53fc6 creating new one. I1102 12:55:16.211457 3598 tablet_bootstrap.cc:234] Tablet 4cad4a9ff0384a9aa991e073a8c3290f: No bootstrap required, opened a new log I1102 12:55:16.211590 3597 tablet_bootstrap.cc:234] Tablet abb97e69180c4801a70e73034ac53fc6: No bootstrap required, opened a new log I1102 12:55:16.211750 3598 ts_tablet_manager.cc:330] Time spent tablet 4cad4a9ff0384a9aa991e073a8c3290f bootstrap: real 0.005s user 0.004s sys 0.001s I1102 12:55:16.211885 3597 ts_tablet_manager.cc:330] Time spent tablet abb97e69180c4801a70e73034ac53fc6 bootstrap: real 0.005s user 0.004s sys 0.001s I1102 12:55:16.213302 3598 ts_tablet_manager.cc:350] Time spent starting tablet 4cad4a9ff0384a9aa991e073a8c3290f: real 0.002s user 0.001s sys 0.000s I1102 12:55:16.213305 3597 ts_tablet_manager.cc:350] Time spent starting tablet abb97e69180c4801a70e73034ac53fc6: real 0.001s user 0.001s sys 0.000s I1102 12:55:16.213948 3602 tablet_peer.cc:213] Configuration changed for tablet: abb97e69180c4801a70e73034ac53fc6 in peer: f82e6599031d4b1aa8525a5828820575 Changed from: seqno: -1 local: true peers { permanent_uuid: "f82e6599031d4b1aa8525a5828820575" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 58036 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "f82e6599031d4b1aa8525a5828820575" role: LEADER last_known_addr { host: "127.0.0.1" port: 58036 } } I1102 12:55:16.213948 3601 tablet_peer.cc:213] Configuration changed for tablet: 4cad4a9ff0384a9aa991e073a8c3290f in peer: f82e6599031d4b1aa8525a5828820575 Changed from: seqno: -1 local: true peers { permanent_uuid: "f82e6599031d4b1aa8525a5828820575" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 58036 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "f82e6599031d4b1aa8525a5828820575" role: LEADER last_known_addr { host: "127.0.0.1" port: 58036 } } I1102 12:55:16.218320 3518 catalog_manager.cc:1127] Tablet: 4cad4a9ff0384a9aa991e073a8c3290f reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "f82e6599031d4b1aa8525a5828820575" role: LEADER last_known_addr { host: "127.0.0.1" port: 58036 } } I1102 12:55:16.219717 3518 catalog_manager.cc:1127] Tablet: abb97e69180c4801a70e73034ac53fc6 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "f82e6599031d4b1aa8525a5828820575" role: LEADER last_known_addr { host: "127.0.0.1" port: 58036 } } I1102 12:55:16.227211 3523 catalog_manager.cc:437] CreateTable from {real_user=alex, eff_user=} at 127.0.0.1:37162: name: "client-testtb2" schema { columns { name: "key" type: UINT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "int_val" type: UINT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } columns { name: "non_null_with_default" type: UINT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION } } num_replicas: 1 I1102 12:55:16.231031 3568 tablet_service.cc:264] Processing CreateTablet for tablet 04cd76eda50742018df2b99f82823da8 (table=client-testtb2 [id=5848e905244f4efaa87d4bb55dbec070]), range=["", ""] I1102 12:55:16.231963 3568 ts_tablet_manager.cc:460] Registered tablet 04cd76eda50742018df2b99f82823da8 I1102 12:55:16.232025 3597 ts_tablet_manager.cc:325] Bootstrapping tablet: 04cd76eda50742018df2b99f82823da8 I1102 12:55:16.232385 3597 tablet_bootstrap.cc:234] Tablet 04cd76eda50742018df2b99f82823da8: Bootstrap starting. I1102 12:55:16.234211 3597 tablet_bootstrap.cc:331] No previous blocks or log segments found for tablet: 04cd76eda50742018df2b99f82823da8 creating new one. I1102 12:55:16.235038 3597 tablet_bootstrap.cc:234] Tablet 04cd76eda50742018df2b99f82823da8: No bootstrap required, opened a new log I1102 12:55:16.235267 3597 ts_tablet_manager.cc:330] Time spent tablet 04cd76eda50742018df2b99f82823da8 bootstrap: real 0.003s user 0.002s sys 0.000s I1102 12:55:16.236248 3597 ts_tablet_manager.cc:350] Time spent starting tablet 04cd76eda50742018df2b99f82823da8: real 0.001s user 0.001s sys 0.000s I1102 12:55:16.236520 3608 tablet_peer.cc:213] Configuration changed for tablet: 04cd76eda50742018df2b99f82823da8 in peer: f82e6599031d4b1aa8525a5828820575 Changed from: seqno: -1 local: true peers { permanent_uuid: "f82e6599031d4b1aa8525a5828820575" role: CANDIDATE last_known_addr { host: "127.0.0.1" port: 58036 } } Changed to: seqno: 0 local: true peers { permanent_uuid: "f82e6599031d4b1aa8525a5828820575" role: LEADER last_known_addr { host: "127.0.0.1" port: 58036 } } I1102 12:55:16.242215 3520 catalog_manager.cc:1127] Tablet: 04cd76eda50742018df2b99f82823da8 reported quorum change. New quorum: seqno: 0 local: true peers { permanent_uuid: "f82e6599031d4b1aa8525a5828820575" role: LEADER last_known_addr { host: "127.0.0.1" port: 58036 } } I1102 12:55:16.389113 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:16.404464 31819 ts_tablet_manager.cc:402] Shutting down tablet manager... I1102 12:55:16.404590 31819 maintenance_manager.cc:177] Unregistered op LogGCOp I1102 12:55:16.404665 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 04cd76eda50742018df2b99f82823da8 I1102 12:55:16.404799 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(04cd76eda50742018df2b99f82823da8) I1102 12:55:16.404846 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(04cd76eda50742018df2b99f82823da8) I1102 12:55:16.404877 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(04cd76eda50742018df2b99f82823da8) I1102 12:55:16.405985 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 4cad4a9ff0384a9aa991e073a8c3290f I1102 12:55:16.406041 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(4cad4a9ff0384a9aa991e073a8c3290f) I1102 12:55:16.406069 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(4cad4a9ff0384a9aa991e073a8c3290f) I1102 12:55:16.406092 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(4cad4a9ff0384a9aa991e073a8c3290f) I1102 12:55:16.407392 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: abb97e69180c4801a70e73034ac53fc6 I1102 12:55:16.407444 31819 maintenance_manager.cc:177] Unregistered op FlushMRSOp(abb97e69180c4801a70e73034ac53fc6) I1102 12:55:16.407471 31819 maintenance_manager.cc:177] Unregistered op CompactRowSetsOp(abb97e69180c4801a70e73034ac53fc6) I1102 12:55:16.407495 31819 maintenance_manager.cc:177] Unregistered op FlushDeltaMemStoresOp(abb97e69180c4801a70e73034ac53fc6) I1102 12:55:16.415057 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:16.415076 31819 tablet_server.cc:114] TabletServer shutting down... I1102 12:55:16.415103 31819 tablet_server.cc:123] TabletServer shut down complete. Bye! I1102 12:55:16.415933 31819 master.cc:124] Master@127.0.0.1:51671 shutting down... I1102 12:55:16.430531 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 11111111111111111111111111111111 I1102 12:55:16.431656 31819 tablet_peer.cc:231] Initiating TabletPeer shutdown for tablet: 00000000000000000000000000000000 I1102 12:55:16.432677 31819 master.cc:127] Master@127.0.0.1:51671 shutdown complete. [ OK ] ClientTest.TestMasterLookupPermits (435 ms) [----------] 34 tests from ClientTest (19911 ms total) [----------] Global test environment tear-down [==========] 34 tests from 1 test case ran. (19912 ms total) [ PASSED ] 34 tests. YOU HAVE 2 DISABLED TESTS ================================================================= ==31819==ERROR: LeakSanitizer: detected memory leaks Indirect leak of 583 byte(s) in 9 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? Indirect leak of 576 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x209043d in kudu::consensus::Peer::NewRemotePeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, gscoped_ptr >, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:221 #2 0x209ea8e in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:443 (discriminator 4) #3 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #4 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #5 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #6 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #7 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #8 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #9 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #10 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #11 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #12 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #13 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #14 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #15 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #16 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #17 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #18 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #19 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #20 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #21 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #22 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #23 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #24 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #25 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 576 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x36e484e in kudu::rpc::Proxy::AsyncRequest(std::basic_string, std::allocator > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, boost::function const&) const /srv/home/alex/dev/kudu/src/kudu/rpc/proxy.cc:65 #2 0x32e39b6 in kudu::consensus::ConsensusServiceProxy::RequestConsensusVoteAsync(kudu::consensus::VoteRequestPB const&, kudu::consensus::VoteResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.proxy.cc:60 (discriminator 1) #3 0x20a3c0e in kudu::consensus::RpcPeerProxy::RequestConsensusVoteAsync(kudu::consensus::VoteRequestPB const*, kudu::consensus::VoteResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:508 #4 0x218e198 in kudu::consensus::LeaderElection::Run() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:184 (discriminator 1) #5 0x21fe247 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:197 (discriminator 1) #6 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #7 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #8 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #9 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #10 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #11 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #12 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #13 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #14 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #15 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 576 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x36e484e in kudu::rpc::Proxy::AsyncRequest(std::basic_string, std::allocator > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, boost::function const&) const /srv/home/alex/dev/kudu/src/kudu/rpc/proxy.cc:65 #2 0x32e2c66 in kudu::consensus::ConsensusServiceProxy::UpdateConsensusAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.proxy.cc:49 (discriminator 1) #3 0x20a378e in kudu::consensus::RpcPeerProxy::UpdateAsync(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:499 #4 0x20d94ed in kudu::consensus::RemotePeer::SendRequest(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::Callback const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:177 (discriminator 3) #5 0x209413e in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:311 #6 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #7 0x20d3ad4 in boost::_mfi::mf1::operator()(kudu::consensus::Peer*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #8 0x20d351b in kudu::Status boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, long) /usr/include/boost/bind/bind.hpp:303 #9 0x20d3154 in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #10 0x20d1fb3 in boost::detail::function::function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, kudu::Status>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:132 #11 0x3763a35 in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #12 0x3b98620 in kudu::ResettableHeartbeaterThread::RunThread() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:106 #13 0x3b9c9d9 in boost::_mfi::mf0::operator()(kudu::ResettableHeartbeaterThread*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #14 0x3b9c535 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #15 0x3b9c21a in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #16 0x3b9b104 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #17 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #18 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #19 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 520 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21f205e in RaftConsensus /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:64 #2 0x18b1787 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:134 #3 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #4 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #5 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #6 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #7 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #8 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #9 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #10 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #11 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #12 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #13 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #17 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 400 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x18b0f40 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 #2 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #3 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #4 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #5 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #6 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #7 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #8 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #9 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #10 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #11 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #12 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #13 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #14 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #15 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #16 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 400 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3bd0a4c in kudu::Thread::StartThread(std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, boost::function const&, scoped_refptr*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:365 #2 0x3b999db in kudu::Status kudu::Thread::Create(std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, void (kudu::ResettableHeartbeaterThread::* const&)(), kudu::ResettableHeartbeaterThread* const&, scoped_refptr*) /srv/home/alex/dev/kudu/src/kudu/util/thread.h:123 #3 0x3b95088 in kudu::ResettableHeartbeaterThread::Start() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:122 (discriminator 4) #4 0x3b9470d in kudu::ResettableHeartbeater::Start() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:61 #5 0x208f948 in kudu::consensus::Peer::Init() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:258 (discriminator 1) #6 0x209079d in kudu::consensus::Peer::NewRemotePeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, gscoped_ptr >, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:226 (discriminator 2) #7 0x209ea8e in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:443 (discriminator 4) #8 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #9 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #10 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #11 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #12 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #13 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #14 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #15 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #16 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #17 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #18 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #19 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #20 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #21 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #22 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #23 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #24 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #25 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #26 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #27 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #28 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #29 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) Indirect leak of 368 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3b94138 in ResettableHeartbeater /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:57 #2 0x2091e4d in Peer /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:246 (discriminator 9) #3 0x2090674 in kudu::consensus::Peer::NewRemotePeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, gscoped_ptr >, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:221 (discriminator 1) #4 0x209ea8e in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:443 (discriminator 4) #5 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #6 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #7 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #8 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #9 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #10 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #11 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #12 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #13 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #14 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #15 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #16 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #17 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #18 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #19 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #20 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #21 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #22 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #23 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #24 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #25 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #26 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #27 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 312 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3c17912 in kudu::ThreadPoolBuilder::Build(gscoped_ptr >*) const /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:77 #2 0x21f0f2d in RaftConsensus /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:63 (discriminator 4) #3 0x18b1787 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:134 #4 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #5 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #6 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #7 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #8 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #9 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #10 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #11 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #12 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #13 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #14 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #15 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #16 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #17 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #18 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 312 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3c17912 in kudu::ThreadPoolBuilder::Build(gscoped_ptr >*) const /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:77 #2 0x2130afd in AsyncLogReader /srv/home/alex/dev/kudu/src/kudu/consensus/async_log_reader.h:39 (discriminator 3) #3 0x2104912 in kudu::consensus::PeerMessageQueue::Init(kudu::consensus::RaftConsensusQueueIface*, kudu::consensus::OpId const&, unsigned long, int) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:142 (discriminator 3) #4 0x2204565 in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:229 #5 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #6 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #7 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #8 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #9 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #10 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #11 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #12 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #13 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #14 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #15 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #16 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #17 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #18 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #19 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #20 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #21 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #22 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #23 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #24 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #25 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #26 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 288 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x208dc68 in kudu::consensus::Peer::NewLocalPeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, kudu::log::Log*, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:203 #2 0x209d050 in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:429 (discriminator 1) #3 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #4 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #5 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #6 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #7 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #8 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #9 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #10 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #11 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #12 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #13 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #14 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #15 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #16 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #17 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #18 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #19 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #20 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #21 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #22 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #23 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #24 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #25 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 220 byte(s) in 2 object(s) allocated from: #0 0xafd1fb in operator new[](unsigned long) ??:? #1 0x3ba7e3e in kudu::Status::CopyState(char const*) /srv/home/alex/dev/kudu/src/kudu/util/status.cc:17 #2 0xc90971 in kudu::Status::operator=(kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/util/status.h:297 (discriminator 2) #3 0x3693d51 in kudu::rpc::OutboundCall::SetFailed(kudu::Status const&, kudu::rpc::ErrorStatusPB*) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:194 #4 0x37ad393 in kudu::rpc::Connection::Shutdown(kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:124 #5 0x36fda77 in kudu::rpc::ReactorThread::DestroyConnection(kudu::rpc::Connection*, kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/rpc/reactor.cc:414 #6 0x3709d0e in kudu::rpc::ReactorThread::CompleteConnectionNegotiation(scoped_refptr const&, kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/rpc/reactor.cc:355 #7 0x37ff050 in kudu::rpc::NegotiationCompletedTask::Run(kudu::rpc::ReactorThread*) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:544 #8 0x36fb25a in kudu::rpc::ReactorThread::AsyncHandler(ev::async&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/reactor.cc:172 #9 0x375e2ae in void ev::base::method_thunk(ev_loop*, ev_async*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 12) #10 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 208 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3a8778f in kudu::MemTracker::FindOrCreateTracker(long, std::basic_string, std::allocator > const&, kudu::MemTracker*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:136 #2 0x21024b7 in PeerMessageQueue /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:112 #3 0x18b103a in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 (discriminator 1) #4 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #5 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #6 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #7 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #8 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #9 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #10 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #11 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #12 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #13 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #14 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #15 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #16 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #17 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #18 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 208 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3a80073 in kudu::MemTracker::CreateTracker(long, std::basic_string, std::allocator > const&, kudu::MemTracker*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:47 #2 0x2102860 in PeerMessageQueue /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:116 #3 0x18b103a in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 (discriminator 1) #4 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #5 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #6 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #7 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #8 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #9 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #10 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #11 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #12 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #13 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #14 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #15 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #16 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #17 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #18 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 208 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x218a919 in LeaderElection /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:151 #2 0x21fe041 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:192 #3 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #4 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #5 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #6 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #7 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #8 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #9 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #10 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #11 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #12 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 200 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3bd0a4c in kudu::Thread::StartThread(std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, boost::function const&, scoped_refptr*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:365 #2 0x3b999db in kudu::Status kudu::Thread::Create(std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, void (kudu::ResettableHeartbeaterThread::* const&)(), kudu::ResettableHeartbeaterThread* const&, scoped_refptr*) /srv/home/alex/dev/kudu/src/kudu/util/thread.h:123 #3 0x3b95088 in kudu::ResettableHeartbeaterThread::Start() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:122 (discriminator 4) #4 0x3b9470d in kudu::ResettableHeartbeater::Start() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:61 #5 0x208f948 in kudu::consensus::Peer::Init() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:258 (discriminator 1) #6 0x208df6f in kudu::consensus::Peer::NewLocalPeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, kudu::log::Log*, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:208 (discriminator 2) #7 0x209d050 in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:429 (discriminator 1) #8 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #9 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #10 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #11 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #12 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #13 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #14 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #15 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #16 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #17 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #18 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #19 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #20 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #21 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #22 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #23 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #24 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #25 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #26 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #27 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #28 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #29 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) Indirect leak of 184 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3b94138 in ResettableHeartbeater /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:57 #2 0x2091e4d in Peer /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:246 (discriminator 9) #3 0x208de9f in kudu::consensus::Peer::NewLocalPeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, kudu::log::Log*, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:203 (discriminator 1) #4 0x209d050 in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:429 (discriminator 1) #5 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #6 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #7 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #8 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #9 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #10 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #11 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #12 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #13 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #14 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #15 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #16 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #17 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #18 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #19 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #20 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #21 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #22 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #23 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #24 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #25 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #26 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #27 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 184 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21fdc9a in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:192 #2 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #3 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #4 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #5 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #6 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #7 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #8 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #9 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #10 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #11 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 171 byte(s) in 3 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x35b05d5 in kudu::metadata::QuorumPeerPB::MergeFrom(kudu::metadata::QuorumPeerPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:425 #3 0x11a6da5 in google::protobuf::internal::GenericTypeHandler::Merge(kudu::metadata::QuorumPeerPB const&, kudu::metadata::QuorumPeerPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:339 #4 0x11a6b67 in void google::protobuf::internal::RepeatedPtrFieldBase::MergeFrom::TypeHandler>(google::protobuf::internal::RepeatedPtrFieldBase const&) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:866 #5 0x35c45bd in google::protobuf::RepeatedPtrField::MergeFrom(google::protobuf::RepeatedPtrField const&) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:1091 #6 0x35b9a4a in kudu::metadata::QuorumPB::MergeFrom(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:714 #7 0x321466d in kudu::consensus::ChangeConfigRequestPB::MergeFrom(kudu::consensus::ChangeConfigRequestPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:1897 #8 0x3223967 in kudu::consensus::ReplicateMsg::MergeFrom(kudu::consensus::ReplicateMsg const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:2553 #9 0x3222e44 in ReplicateMsg /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:2190 #10 0x221d1c0 in kudu::consensus::RaftConsensus::UpdateReplica(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:591 #11 0x2214b7f in kudu::consensus::RaftConsensus::Update(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:419 (discriminator 1) #12 0x142365b in kudu::tserver::ConsensusServiceImpl::UpdateConsensus(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:441 (discriminator 2) #13 0x32de550 in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:76 (discriminator 1) #14 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #15 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #16 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #17 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #18 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #19 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #20 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #21 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 166 byte(s) in 2 object(s) allocated from: #0 0xafd1fb in operator new[](unsigned long) ??:? #1 0x39f0bbf in kudu::faststring::GrowArray(unsigned long) /srv/home/alex/dev/kudu/src/kudu/util/faststring.cc:27 #2 0x1a25616 in kudu::faststring::reserve(unsigned long) /srv/home/alex/dev/kudu/src/kudu/util/faststring.h:79 #3 0x1a2533c in kudu::faststring::resize(unsigned long) /srv/home/alex/dev/kudu/src/kudu/util/faststring.h:55 #4 0x37775e6 in kudu::rpc::serialization::SerializeMessage(google::protobuf::MessageLite const&, kudu::faststring*) /srv/home/alex/dev/kudu/src/kudu/rpc/serialization.cc:42 #5 0x368f156 in kudu::rpc::OutboundCall::SetRequestParam(google::protobuf::Message const&) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:70 #6 0x36e4a77 in kudu::rpc::Proxy::AsyncRequest(std::basic_string, std::allocator > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, boost::function const&) const /srv/home/alex/dev/kudu/src/kudu/rpc/proxy.cc:67 #7 0x32e2c66 in kudu::consensus::ConsensusServiceProxy::UpdateConsensusAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.proxy.cc:49 (discriminator 1) #8 0x20a378e in kudu::consensus::RpcPeerProxy::UpdateAsync(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:499 #9 0x20d94ed in kudu::consensus::RemotePeer::SendRequest(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::Callback const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:177 (discriminator 3) #10 0x209413e in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:311 #11 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #12 0x20d3ad4 in boost::_mfi::mf1::operator()(kudu::consensus::Peer*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #13 0x20d351b in kudu::Status boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, long) /usr/include/boost/bind/bind.hpp:303 #14 0x20d3154 in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #15 0x20d1fb3 in boost::detail::function::function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, kudu::Status>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:132 #16 0x3763a35 in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #17 0x3b98620 in kudu::ResettableHeartbeaterThread::RunThread() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:106 #18 0x3b9c9d9 in boost::_mfi::mf0::operator()(kudu::ResettableHeartbeaterThread*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #19 0x3b9c535 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #20 0x3b9c21a in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #21 0x3b9b104 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #22 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #23 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #24 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 158 byte(s) in 2 object(s) allocated from: #0 0xafd1fb in operator new[](unsigned long) ??:? #1 0x39f0bbf in kudu::faststring::GrowArray(unsigned long) /srv/home/alex/dev/kudu/src/kudu/util/faststring.cc:27 #2 0x1a25616 in kudu::faststring::reserve(unsigned long) /srv/home/alex/dev/kudu/src/kudu/util/faststring.h:79 #3 0x1a2533c in kudu::faststring::resize(unsigned long) /srv/home/alex/dev/kudu/src/kudu/util/faststring.h:55 #4 0x37775e6 in kudu::rpc::serialization::SerializeMessage(google::protobuf::MessageLite const&, kudu::faststring*) /srv/home/alex/dev/kudu/src/kudu/rpc/serialization.cc:42 #5 0x368f156 in kudu::rpc::OutboundCall::SetRequestParam(google::protobuf::Message const&) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:70 #6 0x36e4a77 in kudu::rpc::Proxy::AsyncRequest(std::basic_string, std::allocator > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, boost::function const&) const /srv/home/alex/dev/kudu/src/kudu/rpc/proxy.cc:67 #7 0x32e39b6 in kudu::consensus::ConsensusServiceProxy::RequestConsensusVoteAsync(kudu::consensus::VoteRequestPB const&, kudu::consensus::VoteResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.proxy.cc:60 (discriminator 1) #8 0x20a3c0e in kudu::consensus::RpcPeerProxy::RequestConsensusVoteAsync(kudu::consensus::VoteRequestPB const*, kudu::consensus::VoteResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:508 #9 0x218e198 in kudu::consensus::LeaderElection::Run() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:184 (discriminator 1) #10 0x21fe247 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:197 (discriminator 1) #11 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #12 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #13 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #14 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #15 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #16 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #17 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #18 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #19 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #20 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 152 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x18b1326 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:134 #2 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #3 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #4 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #5 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #6 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #7 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #8 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #9 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #10 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #11 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #12 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #13 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #14 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #15 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #16 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 152 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x210482b in kudu::consensus::PeerMessageQueue::Init(kudu::consensus::RaftConsensusQueueIface*, kudu::consensus::OpId const&, unsigned long, int) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:142 #2 0x2204565 in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:229 #3 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #4 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #5 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #6 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #7 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #8 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #9 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #10 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #11 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #12 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #13 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #14 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #15 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #16 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #17 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #18 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #19 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #20 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #21 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #22 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #23 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #24 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 144 byte(s) in 3 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x11a7db8 in google::protobuf::internal::GenericTypeHandler::New() /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:335 #2 0x11a7ac5 in google::protobuf::RepeatedPtrField::TypeHandler::Type* google::protobuf::internal::RepeatedPtrFieldBase::Add::TypeHandler>() /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:843 #3 0x127132b in google::protobuf::RepeatedPtrField::Add() /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:1048 #4 0x1140ef2 in kudu::metadata::QuorumPB::add_peers() /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.h:507 #5 0x35bce00 in kudu::metadata::QuorumPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:588 #6 0x3117df7 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::metadata::QuorumPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #7 0x320d441 in kudu::consensus::ConsensusMetadataPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:1411 #8 0x3eaa5ca in google::protobuf::(anonymous namespace)::InlineMergeFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/message_lite.cc:121 #9 0x208b3c0 in kudu::consensus::ConsensusMetadata::Load(kudu::FsManager*, std::basic_string, std::allocator > const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_meta.cc:41 (discriminator 5) #10 0x18b0503 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:108 (discriminator 3) #11 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #12 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #13 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #14 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #15 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #16 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #17 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #18 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #19 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #20 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #21 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #22 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #23 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #24 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #25 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 144 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20a6a2c in kudu::consensus::(anonymous namespace)::CreateConsensusServiceProxyForHost(std::tr1::shared_ptr const&, kudu::HostPort const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:526 #2 0x20a514f in kudu::consensus::RpcPeerProxyFactory::NewProxy(kudu::metadata::QuorumPeerPB const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:541 (discriminator 2) #3 0x209e19c in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:439 (discriminator 1) #4 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #5 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #6 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #7 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #8 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #9 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #10 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #11 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #12 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #13 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #14 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #15 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #16 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #17 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #18 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #19 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #20 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #21 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #22 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #23 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #24 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #25 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #26 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 144 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20a6a2c in kudu::consensus::(anonymous namespace)::CreateConsensusServiceProxyForHost(std::tr1::shared_ptr const&, kudu::HostPort const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:526 #2 0x20a514f in kudu::consensus::RpcPeerProxyFactory::NewProxy(kudu::metadata::QuorumPeerPB const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:541 (discriminator 2) #3 0x21f656a in kudu::consensus::RaftConsensus::Start(kudu::consensus::ConsensusBootstrapInfo const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:111 (discriminator 2) #4 0x18b4e50 in kudu::tablet::TabletPeer::Start(kudu::consensus::ConsensusBootstrapInfo const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:176 (discriminator 1) #5 0x1486d90 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:364 (discriminator 1) #6 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #7 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #8 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #9 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #10 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #11 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #12 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #13 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #14 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #15 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #16 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #17 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #18 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #19 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 135 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x7f7f2ca1bef7 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) ??:? #3 0x7f7ef188f8ff () #4 0x7f7ef188f95f () #5 0x7f7e00000002 () Indirect leak of 134 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x3bd0b1f in kudu::Thread::StartThread(std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, boost::function const&, scoped_refptr*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:365 #3 0x3b999db in kudu::Status kudu::Thread::Create(std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, void (kudu::ResettableHeartbeaterThread::* const&)(), kudu::ResettableHeartbeaterThread* const&, scoped_refptr*) /srv/home/alex/dev/kudu/src/kudu/util/thread.h:123 #4 0x3b95088 in kudu::ResettableHeartbeaterThread::Start() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:122 (discriminator 4) #5 0x3b9470d in kudu::ResettableHeartbeater::Start() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:61 #6 0x208f948 in kudu::consensus::Peer::Init() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:258 (discriminator 1) #7 0x209079d in kudu::consensus::Peer::NewRemotePeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, gscoped_ptr >, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:226 (discriminator 2) #8 0x209ea8e in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:443 (discriminator 4) #9 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #10 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #11 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #12 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #13 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #14 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #15 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #16 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #17 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #18 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #19 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #20 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #21 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #22 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #23 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #24 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #25 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #26 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #27 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #28 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #29 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) Indirect leak of 120 byte(s) in 3 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x1141a16 in kudu::metadata::QuorumPeerPB::mutable_last_known_addr() /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.h:426 (discriminator 1) #2 0x35b3ce1 in kudu::metadata::QuorumPeerPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:293 #3 0x35c3ce7 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::metadata::QuorumPeerPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #4 0x35bce10 in kudu::metadata::QuorumPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:588 #5 0x3117df7 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::metadata::QuorumPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #6 0x320d441 in kudu::consensus::ConsensusMetadataPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:1411 #7 0x3eaa5ca in google::protobuf::(anonymous namespace)::InlineMergeFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/message_lite.cc:121 #8 0x208b3c0 in kudu::consensus::ConsensusMetadata::Load(kudu::FsManager*, std::basic_string, std::allocator > const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_meta.cc:41 (discriminator 5) #9 0x18b0503 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:108 (discriminator 3) #10 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #11 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #12 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #13 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #17 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #18 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #19 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #20 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #21 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #22 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #23 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #24 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 120 byte(s) in 3 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x2134266 in kudu::consensus::ConsensusRequestPB::mutable_committed_index() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:4353 (discriminator 1) #2 0x21148f7 in kudu::consensus::PeerMessageQueue::RequestForPeer(std::basic_string, std::allocator > const&, kudu::consensus::ConsensusRequestPB*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:278 #3 0x2093607 in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:289 #4 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #5 0x20a0d87 in kudu::consensus::PeerManager::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:467 (discriminator 1) #6 0x220b6bb in kudu::consensus::RaftConsensus::Replicate(kudu::consensus::ConsensusRound*) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:303 (discriminator 1) #7 0x191990d in kudu::tablet::TransactionDriver::PrepareAndStart() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:159 #8 0x1917012 in kudu::tablet::TransactionDriver::PrepareAndStartTask() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:114 #9 0x192ad39 in kudu::internal::RunnableAdapter::Run(kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:134 (discriminator 4) #10 0x192a908 in kudu::internal::InvokeHelper, void (kudu::tablet::TransactionDriver*)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:871 #11 0x192a58c in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::tablet::TransactionDriver*), void (kudu::internal::UnretainedWrapper)>, void (kudu::tablet::TransactionDriver*)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1063 #12 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #13 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #14 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #15 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #16 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #17 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #18 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #19 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #20 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #21 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #22 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #23 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #24 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #25 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #26 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 120 byte(s) in 3 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x2135606 in kudu::consensus::ConsensusRequestPB::mutable_preceding_id() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:4315 (discriminator 1) #2 0x21156b4 in kudu::consensus::PeerMessageQueue::RequestForPeer(std::basic_string, std::allocator > const&, kudu::consensus::ConsensusRequestPB*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:323 #3 0x2093607 in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:289 #4 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #5 0x20a0d87 in kudu::consensus::PeerManager::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:467 (discriminator 1) #6 0x220b6bb in kudu::consensus::RaftConsensus::Replicate(kudu::consensus::ConsensusRound*) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:303 (discriminator 1) #7 0x191990d in kudu::tablet::TransactionDriver::PrepareAndStart() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:159 #8 0x1917012 in kudu::tablet::TransactionDriver::PrepareAndStartTask() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:114 #9 0x192ad39 in kudu::internal::RunnableAdapter::Run(kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:134 (discriminator 4) #10 0x192a908 in kudu::internal::InvokeHelper, void (kudu::tablet::TransactionDriver*)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:871 #11 0x192a58c in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::tablet::TransactionDriver*), void (kudu::internal::UnretainedWrapper)>, void (kudu::tablet::TransactionDriver*)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1063 #12 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #13 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #14 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #15 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #16 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #17 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #18 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #19 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #20 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #21 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #22 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #23 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #24 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #25 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #26 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 114 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x35b05d5 in kudu::metadata::QuorumPeerPB::MergeFrom(kudu::metadata::QuorumPeerPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:425 #3 0x11a6da5 in google::protobuf::internal::GenericTypeHandler::Merge(kudu::metadata::QuorumPeerPB const&, kudu::metadata::QuorumPeerPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:339 #4 0x11a6b67 in void google::protobuf::internal::RepeatedPtrFieldBase::MergeFrom::TypeHandler>(google::protobuf::internal::RepeatedPtrFieldBase const&) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:866 #5 0x35c45bd in google::protobuf::RepeatedPtrField::MergeFrom(google::protobuf::RepeatedPtrField const&) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:1091 #6 0x35b9a4a in kudu::metadata::QuorumPB::MergeFrom(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:714 #7 0x35c1d48 in kudu::metadata::QuorumPB::CopyFrom(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:735 #8 0x21f5295 in kudu::consensus::RaftConsensus::Start(kudu::consensus::ConsensusBootstrapInfo const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:95 (discriminator 2) #9 0x18b4e50 in kudu::tablet::TabletPeer::Start(kudu::consensus::ConsensusBootstrapInfo const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:176 (discriminator 1) #10 0x1486d90 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:364 (discriminator 1) #11 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #12 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #13 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #17 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #18 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #19 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #20 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #21 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #22 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #23 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #24 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 112 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x7f7f2ca1bef7 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) ??:? #3 0x7f7f0c33feff () #3 0x3f62ff3 in ?? ??:0 Indirect leak of 112 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x3a80131 in kudu::MemTracker::CreateTracker(long, std::basic_string, std::allocator > const&, kudu::MemTracker*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:47 #3 0x2102860 in PeerMessageQueue /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:116 #4 0x18b103a in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 (discriminator 1) #5 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #6 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #7 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #8 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #9 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #10 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #11 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #12 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #13 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #14 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #15 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #16 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #17 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #18 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #19 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 112 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x7f7f2ca1bef7 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) ??:? #3 0x7f7ef188fe9f () #3 0x3f62ff3 in ?? ??:0 Indirect leak of 104 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x18b10dd in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:127 #2 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #3 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #4 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #5 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #6 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #7 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #8 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #9 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #10 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #11 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #12 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #13 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #14 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #15 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #16 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 102 byte(s) in 3 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x3667ea5 in kudu::HostPortPB::MergeFrom(kudu::HostPortPB const&) /srv/home/alex/dev/kudu/src/kudu/common/common.pb.cc:1279 #3 0x35b07ed in kudu::metadata::QuorumPeerPB::MergeFrom(kudu::metadata::QuorumPeerPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:431 #4 0x11a6da5 in google::protobuf::internal::GenericTypeHandler::Merge(kudu::metadata::QuorumPeerPB const&, kudu::metadata::QuorumPeerPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:339 #5 0x11a6b67 in void google::protobuf::internal::RepeatedPtrFieldBase::MergeFrom::TypeHandler>(google::protobuf::internal::RepeatedPtrFieldBase const&) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:866 #6 0x35c45bd in google::protobuf::RepeatedPtrField::MergeFrom(google::protobuf::RepeatedPtrField const&) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:1091 #7 0x35b9a4a in kudu::metadata::QuorumPB::MergeFrom(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:714 #8 0x321466d in kudu::consensus::ChangeConfigRequestPB::MergeFrom(kudu::consensus::ChangeConfigRequestPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:1897 #9 0x3223967 in kudu::consensus::ReplicateMsg::MergeFrom(kudu::consensus::ReplicateMsg const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:2553 #10 0x3222e44 in ReplicateMsg /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:2190 #11 0x221d1c0 in kudu::consensus::RaftConsensus::UpdateReplica(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:591 #12 0x2214b7f in kudu::consensus::RaftConsensus::Update(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:419 (discriminator 1) #13 0x142365b in kudu::tserver::ConsensusServiceImpl::UpdateConsensus(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:441 (discriminator 2) #14 0x32de550 in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:76 (discriminator 1) #15 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #16 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #17 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #18 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #19 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #20 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #21 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #22 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 96 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0xe48b06 in __gnu_cxx::new_allocator, std::allocator >, false>*>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0xe46eb3 in std::tr1::_Hashtable, std::allocator >, std::basic_string, std::allocator >, std::allocator, std::allocator > >, std::_Identity, std::allocator > >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, true, true>::_M_allocate_buckets(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:493 #3 0x22542cb in _Hashtable /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:593 #4 0x2253dcd in __unordered_set /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/unordered_set.h:43 #5 0x2253bdd in unordered_set /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/unordered_set.h:173 #6 0x2257c2f in QuorumState /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus_state.cc:86 #7 0x225712e in kudu::consensus::QuorumState::Build(kudu::metadata::QuorumPB const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus_state.cc:70 #8 0x2259fce in kudu::consensus::ReplicaState::ResetActiveQuorumStateUnlocked(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus_state.cc:659 #9 0x2262b01 in kudu::consensus::ReplicaState::SetPendingQuorumUnlocked(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus_state.cc:260 #10 0x2202c96 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1024 (discriminator 2) #11 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #12 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #13 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #14 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #15 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #16 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #17 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #18 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #19 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #20 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #21 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #22 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #23 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #24 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #25 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #26 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #27 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #28 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #29 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) Indirect leak of 96 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21b93a6 in __gnu_cxx::new_allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*>, false>*>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x21b7d13 in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::LeaderElection::VoterState*>, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> >, std::_Select1st, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_allocate_buckets(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:493 #3 0x21c694d in _Hashtable /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:533 #4 0x21c61be in __unordered_map /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/unordered_map.h:73 #5 0x219b79b in unordered_map /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/unordered_map.h:198 #6 0x21896b0 in LeaderElection /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:147 (discriminator 7) #7 0x21fe041 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:192 #8 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #9 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #10 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #11 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #12 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #13 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #17 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 96 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21a1197 in _ZN4kudu4BindIMNS_9consensus14LeaderElectionEFvRKSsEPS2_SsEENS_8CallbackINS_8internal9BindStateINS9_13FunctorTraitsIT_E12RunnableTypeENSD_7RunTypeEFvNS9_19CallbackParamTraitsIT0_Xsr14IsMoveOnlyTypeISH_EE5valueEE11StorageTypeENSG_IT1_Xsr14IsMoveOnlyTypeISK_EE5valueEE11StorageTypeEEE14UnboundRunTypeEEESC_RKSH_RKSK_ /srv/home/alex/dev/kudu/src/kudu/gutil/bind.h:171 #2 0x218df33 in kudu::consensus::LeaderElection::Run() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:191 #3 0x21fe247 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:197 (discriminator 1) #4 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #5 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #6 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #7 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #8 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #9 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #10 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #11 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #12 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #13 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 96 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20c83c6 in __gnu_cxx::new_allocator, std::allocator > const, kudu::consensus::Peer*>, false>*>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x20c6773 in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::Peer*>, std::allocator, std::allocator > const, kudu::consensus::Peer*> >, std::_Select1st, std::allocator > const, kudu::consensus::Peer*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_allocate_buckets(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:493 #3 0x20cc2ad in _Hashtable /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:533 #4 0x20cbb1e in __unordered_map /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/unordered_map.h:73 #5 0x20b3e9b in unordered_map /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/unordered_map.h:198 #6 0x2099cc7 in PeerManager /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:406 (discriminator 1) #7 0x18b1268 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:127 #8 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #9 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #10 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #11 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #12 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #13 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #14 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #15 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #16 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #17 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #18 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #19 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #20 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #21 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #22 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 96 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21568e6 in __gnu_cxx::new_allocator, std::allocator > const, kudu::consensus::PeerMessageQueue::TrackedPeer*>, false>*>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x21541f3 in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::PeerMessageQueue::TrackedPeer*>, std::allocator, std::allocator > const, kudu::consensus::PeerMessageQueue::TrackedPeer*> >, std::_Select1st, std::allocator > const, kudu::consensus::PeerMessageQueue::TrackedPeer*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_allocate_buckets(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:493 #3 0x217f18d in _Hashtable /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:533 #4 0x217e9fe in __unordered_map /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/unordered_map.h:73 #5 0x212ee9b in unordered_map /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/unordered_map.h:198 #6 0x2102297 in PeerMessageQueue /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:105 #7 0x18b103a in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 (discriminator 1) #8 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #9 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #10 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #11 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #12 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #13 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #14 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #15 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #16 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #17 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #18 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #19 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #20 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #21 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #22 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 96 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21c4f16 in __gnu_cxx::new_allocator, std::allocator > const, kudu::consensus::PeerProxy*>, false>*>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x21c36d3 in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::PeerProxy*>, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> >, std::_Select1st, std::allocator > const, kudu::consensus::PeerProxy*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_allocate_buckets(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:493 #3 0x22512fd in _Hashtable /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:533 #4 0x2250b6e in __unordered_map /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/unordered_map.h:73 #5 0x223868b in unordered_map /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/unordered_map.h:198 #6 0x21f0e72 in RaftConsensus /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:58 (discriminator 10) #7 0x18b1787 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:134 #8 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #9 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #10 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #11 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #12 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #13 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #14 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #15 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #16 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #17 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #18 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #19 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #20 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #21 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #22 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 96 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x2090267 in kudu::consensus::Peer::NewRemotePeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, gscoped_ptr >, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:220 #2 0x209ea8e in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:443 (discriminator 4) #3 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #4 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #5 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #6 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #7 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #8 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #9 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #10 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #11 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #12 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #13 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #14 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #15 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #16 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #17 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #18 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #19 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #20 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #21 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #22 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #23 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #24 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #25 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 80 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x2256f8b in kudu::consensus::QuorumState::Build(kudu::metadata::QuorumPB const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus_state.cc:70 #2 0x2259fce in kudu::consensus::ReplicaState::ResetActiveQuorumStateUnlocked(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus_state.cc:659 #3 0x2262b01 in kudu::consensus::ReplicaState::SetPendingQuorumUnlocked(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus_state.cc:260 #4 0x2202c96 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1024 (discriminator 2) #5 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #6 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #7 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #8 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #9 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #10 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #11 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #12 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #13 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #14 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #15 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #16 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #17 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #18 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #19 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #20 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #21 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #22 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #23 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #24 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #25 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 80 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x1141a16 in kudu::metadata::QuorumPeerPB::mutable_last_known_addr() /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.h:426 (discriminator 1) #2 0x35b07ae in kudu::metadata::QuorumPeerPB::MergeFrom(kudu::metadata::QuorumPeerPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:431 #3 0x35afe84 in QuorumPeerPB /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:187 #4 0x2091895 in Peer /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:246 (discriminator 1) #5 0x2090674 in kudu::consensus::Peer::NewRemotePeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, gscoped_ptr >, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:221 (discriminator 1) #6 0x209ea8e in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:443 (discriminator 4) #7 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #8 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #9 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #10 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #11 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #12 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #13 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #14 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #15 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #16 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #17 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #18 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #19 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #20 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #21 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #22 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #23 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #24 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #25 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #26 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #27 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #28 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #29 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 80 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20b1a25 in _ZN4kudu4BindIMNS_9consensus4PeerEFvRKNS_6StatusEENS_8internal17UnretainedWrapperIS2_EEEENS_8CallbackINS8_9BindStateINS8_13FunctorTraitsIT_E12RunnableTypeENSF_7RunTypeEFvNS8_19CallbackParamTraitsIT0_Xsr14IsMoveOnlyTypeISJ_EE5valueEE11StorageTypeEEE14UnboundRunTypeEEESE_RKSJ_ /srv/home/alex/dev/kudu/src/kudu/gutil/bind.h:118 #2 0x2094114 in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:312 #3 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #4 0x20d3ad4 in boost::_mfi::mf1::operator()(kudu::consensus::Peer*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #5 0x20d351b in kudu::Status boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, long) /usr/include/boost/bind/bind.hpp:303 #6 0x20d3154 in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #7 0x20d1fb3 in boost::detail::function::function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, kudu::Status>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:132 #8 0x3763a35 in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #9 0x3b98620 in kudu::ResettableHeartbeaterThread::RunThread() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:106 #10 0x3b9c9d9 in boost::_mfi::mf0::operator()(kudu::ResettableHeartbeaterThread*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #11 0x3b9c535 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #12 0x3b9c21a in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #13 0x3b9b104 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #14 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #15 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #16 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 80 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20dfc1f in boost::detail::function::functor_manager const&>, boost::_bi::list2, boost::_bi::value > > > >::manager(boost::detail::function::function_buffer const&, boost::detail::function::function_buffer&, boost::detail::function::functor_manager_operation_type, mpl_::bool_) /usr/include/boost/function/function_base.hpp:380 #2 0x20df7fa in boost::detail::function::functor_manager const&>, boost::_bi::list2, boost::_bi::value > > > >::manager(boost::detail::function::function_buffer const&, boost::detail::function::function_buffer&, boost::detail::function::functor_manager_operation_type, boost::detail::function::function_obj_tag) /usr/include/boost/function/function_base.hpp:412 #3 0x20dcea8 in boost::detail::function::functor_manager const&>, boost::_bi::list2, boost::_bi::value > > > >::manage(boost::detail::function::function_buffer const&, boost::detail::function::function_buffer&, boost::detail::function::functor_manager_operation_type) /usr/include/boost/function/function_base.hpp:440 #4 0x193a2ef in boost::function0::assign_to_own(boost::function0 const&) /usr/include/boost/function/function_template.hpp:909 #5 0x1939f75 in function0 /usr/include/boost/function/function_template.hpp:749 #6 0x193846d in function /usr/include/boost/function/function_template.hpp:1089 #7 0x368a14d in OutboundCall /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:37 (discriminator 3) #8 0x36e4975 in kudu::rpc::Proxy::AsyncRequest(std::basic_string, std::allocator > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, boost::function const&) const /srv/home/alex/dev/kudu/src/kudu/rpc/proxy.cc:65 #9 0x32e2c66 in kudu::consensus::ConsensusServiceProxy::UpdateConsensusAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.proxy.cc:49 (discriminator 1) #10 0x20a378e in kudu::consensus::RpcPeerProxy::UpdateAsync(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:499 #11 0x20d94ed in kudu::consensus::RemotePeer::SendRequest(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::Callback const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:177 (discriminator 3) #12 0x209413e in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:311 #13 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #14 0x20d3ad4 in boost::_mfi::mf1::operator()(kudu::consensus::Peer*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #15 0x20d351b in kudu::Status boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, long) /usr/include/boost/bind/bind.hpp:303 #16 0x20d3154 in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #17 0x20d1fb3 in boost::detail::function::function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, kudu::Status>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:132 #18 0x3763a35 in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #19 0x3b98620 in kudu::ResettableHeartbeaterThread::RunThread() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:106 #20 0x3b9c9d9 in boost::_mfi::mf0::operator()(kudu::ResettableHeartbeaterThread*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #21 0x3b9c535 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #22 0x3b9c21a in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #23 0x3b9b104 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #24 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #25 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #26 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 72 byte(s) in 3 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3a9b9d3 in __gnu_cxx::new_allocator >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x3a9a67f in std::_List_base >::_M_get_node() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_list.h:334 #3 0x3a99e2f in std::list >::_M_create_node(kudu::MemTracker* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_list.h:484 #4 0x3a98f69 in std::list >::insert(std::_List_iterator, kudu::MemTracker* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/list.tcc:101 #5 0x3a82977 in kudu::MemTracker::AddChildTracker(kudu::MemTracker*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:372 #6 0x3a81a28 in MemTracker /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:76 #7 0x3a80131 in kudu::MemTracker::CreateTracker(long, std::basic_string, std::allocator > const&, kudu::MemTracker*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:47 #8 0x2102860 in PeerMessageQueue /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:116 #9 0x18b103a in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 (discriminator 1) #10 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #11 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #12 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #13 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #17 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #18 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #19 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #20 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #21 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #22 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #23 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #24 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 72 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x7f7f2ca1bef7 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) ??:? #3 0xff06185ffc7 () #4 0x7f7f0c33fe7f () Indirect leak of 70 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x7f7f2ca1bef7 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) ??:? #3 0x7f7ef188f7df () #4 0x7f7ef188f83f () #5 0x7f7e00000002 () Indirect leak of 68 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x3667ea5 in kudu::HostPortPB::MergeFrom(kudu::HostPortPB const&) /srv/home/alex/dev/kudu/src/kudu/common/common.pb.cc:1279 #3 0x35b07ed in kudu::metadata::QuorumPeerPB::MergeFrom(kudu::metadata::QuorumPeerPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:431 #4 0x11a6da5 in google::protobuf::internal::GenericTypeHandler::Merge(kudu::metadata::QuorumPeerPB const&, kudu::metadata::QuorumPeerPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:339 #5 0x11a6b67 in void google::protobuf::internal::RepeatedPtrFieldBase::MergeFrom::TypeHandler>(google::protobuf::internal::RepeatedPtrFieldBase const&) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:866 #6 0x35c45bd in google::protobuf::RepeatedPtrField::MergeFrom(google::protobuf::RepeatedPtrField const&) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:1091 #7 0x35b9a4a in kudu::metadata::QuorumPB::MergeFrom(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:714 #8 0x35c1d48 in kudu::metadata::QuorumPB::CopyFrom(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:735 #9 0x21f5295 in kudu::consensus::RaftConsensus::Start(kudu::consensus::ConsensusBootstrapInfo const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:95 (discriminator 2) #10 0x18b4e50 in kudu::tablet::TabletPeer::Start(kudu::consensus::ConsensusBootstrapInfo const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:176 (discriminator 1) #11 0x1486d90 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:364 (discriminator 1) #12 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #13 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #14 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #15 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #16 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #17 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #18 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #19 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #20 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #21 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #22 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #23 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #24 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #25 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 67 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x3bd0b1f in kudu::Thread::StartThread(std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, boost::function const&, scoped_refptr*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:365 #3 0x3b999db in kudu::Status kudu::Thread::Create(std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, void (kudu::ResettableHeartbeaterThread::* const&)(), kudu::ResettableHeartbeaterThread* const&, scoped_refptr*) /srv/home/alex/dev/kudu/src/kudu/util/thread.h:123 #4 0x3b95088 in kudu::ResettableHeartbeaterThread::Start() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:122 (discriminator 4) #5 0x3b9470d in kudu::ResettableHeartbeater::Start() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:61 #6 0x208f948 in kudu::consensus::Peer::Init() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:258 (discriminator 1) #7 0x208df6f in kudu::consensus::Peer::NewLocalPeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, kudu::log::Log*, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:208 (discriminator 2) #8 0x209d050 in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:429 (discriminator 1) #9 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #10 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #11 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #12 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #13 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #14 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #15 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #16 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #17 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #18 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #19 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #20 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #21 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #22 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #23 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #24 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #25 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #26 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #27 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #28 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #29 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) Indirect leak of 64 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x12d3196 in kudu::consensus::ConsensusMetadataPB::mutable_committed_quorum() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:2627 (discriminator 1) #2 0x320d431 in kudu::consensus::ConsensusMetadataPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:1411 #3 0x3eaa5ca in google::protobuf::(anonymous namespace)::InlineMergeFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/message_lite.cc:121 #4 0x208b3c0 in kudu::consensus::ConsensusMetadata::Load(kudu::FsManager*, std::basic_string, std::allocator > const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_meta.cc:41 (discriminator 5) #5 0x18b0503 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:108 (discriminator 3) #6 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #7 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #8 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #9 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #10 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #11 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #12 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #13 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #14 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #15 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #16 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #17 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #18 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #19 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #20 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 64 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21a7f3f in boost::detail::function::functor_manager >, boost::_bi::list1 > > > >::manager(boost::detail::function::function_buffer const&, boost::detail::function::function_buffer&, boost::detail::function::functor_manager_operation_type, mpl_::bool_) /usr/include/boost/function/function_base.hpp:380 #2 0x21a7b1a in boost::detail::function::functor_manager >, boost::_bi::list1 > > > >::manager(boost::detail::function::function_buffer const&, boost::detail::function::function_buffer&, boost::detail::function::functor_manager_operation_type, boost::detail::function::function_obj_tag) /usr/include/boost/function/function_base.hpp:412 #3 0x21a5648 in boost::detail::function::functor_manager >, boost::_bi::list1 > > > >::manage(boost::detail::function::function_buffer const&, boost::detail::function::function_buffer&, boost::detail::function::functor_manager_operation_type) /usr/include/boost/function/function_base.hpp:440 #4 0x193a2ef in boost::function0::assign_to_own(boost::function0 const&) /usr/include/boost/function/function_template.hpp:909 #5 0x1939f75 in function0 /usr/include/boost/function/function_template.hpp:749 #6 0x193846d in function /usr/include/boost/function/function_template.hpp:1089 #7 0x368a14d in OutboundCall /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:37 (discriminator 3) #8 0x36e4975 in kudu::rpc::Proxy::AsyncRequest(std::basic_string, std::allocator > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, boost::function const&) const /srv/home/alex/dev/kudu/src/kudu/rpc/proxy.cc:65 #9 0x32e39b6 in kudu::consensus::ConsensusServiceProxy::RequestConsensusVoteAsync(kudu::consensus::VoteRequestPB const&, kudu::consensus::VoteResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.proxy.cc:60 (discriminator 1) #10 0x20a3c0e in kudu::consensus::RpcPeerProxy::RequestConsensusVoteAsync(kudu::consensus::VoteRequestPB const*, kudu::consensus::VoteResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:508 #11 0x218e198 in kudu::consensus::LeaderElection::Run() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:184 (discriminator 1) #12 0x21fe247 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:197 (discriminator 1) #13 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #14 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #15 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #16 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #17 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #18 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #19 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #20 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #21 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #22 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 64 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x36ed7dd in __shared_count /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:302 #2 0x36ed3c9 in __shared_ptr /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:554 #3 0x36e7a72 in void std::tr1::__shared_ptr::reset(kudu::rpc::OutboundCall*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:653 #4 0x36e49fe in kudu::rpc::Proxy::AsyncRequest(std::basic_string, std::allocator > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, boost::function const&) const /srv/home/alex/dev/kudu/src/kudu/rpc/proxy.cc:66 #5 0x32e39b6 in kudu::consensus::ConsensusServiceProxy::RequestConsensusVoteAsync(kudu::consensus::VoteRequestPB const&, kudu::consensus::VoteResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.proxy.cc:60 (discriminator 1) #6 0x20a3c0e in kudu::consensus::RpcPeerProxy::RequestConsensusVoteAsync(kudu::consensus::VoteRequestPB const*, kudu::consensus::VoteResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:508 #7 0x218e198 in kudu::consensus::LeaderElection::Run() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:184 (discriminator 1) #8 0x21fe247 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:197 (discriminator 1) #9 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #10 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #11 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #12 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #13 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #14 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #15 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #16 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #17 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #18 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 64 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x36ed7dd in __shared_count /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:302 #2 0x36ed3c9 in __shared_ptr /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:554 #3 0x36e7a72 in void std::tr1::__shared_ptr::reset(kudu::rpc::OutboundCall*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:653 #4 0x36e49fe in kudu::rpc::Proxy::AsyncRequest(std::basic_string, std::allocator > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, boost::function const&) const /srv/home/alex/dev/kudu/src/kudu/rpc/proxy.cc:66 #5 0x32e2c66 in kudu::consensus::ConsensusServiceProxy::UpdateConsensusAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.proxy.cc:49 (discriminator 1) #6 0x20a378e in kudu::consensus::RpcPeerProxy::UpdateAsync(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:499 #7 0x20d94ed in kudu::consensus::RemotePeer::SendRequest(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::Callback const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:177 (discriminator 3) #8 0x209413e in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:311 #9 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #10 0x20d3ad4 in boost::_mfi::mf1::operator()(kudu::consensus::Peer*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #11 0x20d351b in kudu::Status boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, long) /usr/include/boost/bind/bind.hpp:303 #12 0x20d3154 in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #13 0x20d1fb3 in boost::detail::function::function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, kudu::Status>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:132 #14 0x3763a35 in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #15 0x3b98620 in kudu::ResettableHeartbeaterThread::RunThread() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:106 #16 0x3b9c9d9 in boost::_mfi::mf0::operator()(kudu::ResettableHeartbeaterThread*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #17 0x3b9c535 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #18 0x3b9c21a in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #19 0x3b9b104 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #20 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #21 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #22 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 64 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21fc71a in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:176 #2 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #3 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #4 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #5 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #6 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #7 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #8 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #9 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #10 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #11 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 64 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x15f77e6 in ServerBase /srv/home/alex/dev/kudu/src/kudu/server/server_base.cc:51 #2 0x1402133 in TabletServer /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_server.cc:43 (discriminator 1) #3 0x13ba675 in kudu::tserver::MiniTabletServer::Start() /srv/home/alex/dev/kudu/src/kudu/tserver/mini_tablet_server.cc:55 #4 0x1023e51 in kudu::MiniCluster::AddTabletServer() /srv/home/alex/dev/kudu/src/kudu/integration-tests/mini_cluster.cc:179 (discriminator 2) #5 0x101c840 in kudu::MiniCluster::Start() /srv/home/alex/dev/kudu/src/kudu/integration-tests/mini_cluster.cc:78 (discriminator 1) #6 0xca88d4 in kudu::client::ClientTest::SetUp() /srv/home/alex/dev/kudu/src/kudu/client/client-test.cc:95 (discriminator 1) #7 0x3f4b702 in void testing::internal::HandleSehExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2078 #8 0x3f3ed70 in void testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2114 #9 0x3f25432 in testing::Test::Run() /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2146 #10 0x3f262ca in testing::TestInfo::Run() /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2326 #11 0x3f26af6 in testing::TestCase::Run() /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2444 #12 0x3f2ebc6 in testing::internal::UnitTestImpl::RunAllTests() /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:4315 #13 0x3f48a22 in bool testing::internal::HandleSehExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2078 #14 0x3f41100 in bool testing::internal::HandleExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2114 #15 0x3f2e84a in testing::UnitTest::Run() /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:3926 #16 0x1065db0 in RUN_ALL_TESTS() /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/include/gtest/gtest.h:2288 #17 0x1064211 in main /srv/home/alex/dev/kudu/src/kudu/util/test_main.cc:34 #18 0x7f7f2c3a5ec4 in __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:287 Indirect leak of 64 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x208dbb0 in kudu::consensus::Peer::NewLocalPeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, kudu::log::Log*, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:202 #2 0x209d050 in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:429 (discriminator 1) #3 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #4 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #5 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #6 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #7 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #8 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #9 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #10 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #11 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #12 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #13 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #14 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #15 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #16 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #17 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #18 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #19 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #20 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #21 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #22 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #23 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #24 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #25 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 64 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x208b196 in kudu::consensus::ConsensusMetadata::Load(kudu::FsManager*, std::basic_string, std::allocator > const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_meta.cc:40 #2 0x18b0503 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:108 (discriminator 3) #3 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #4 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #5 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #6 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #7 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #8 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #9 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #10 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #11 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #12 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #13 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #17 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 64 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20d4802 in boost::detail::function::functor_manager, boost::_bi::list2, boost::_bi::value > > >::manager(boost::detail::function::function_buffer const&, boost::detail::function::function_buffer&, boost::detail::function::functor_manager_operation_type, mpl_::bool_) /usr/include/boost/function/function_base.hpp:380 #2 0x20d441a in boost::detail::function::functor_manager, boost::_bi::list2, boost::_bi::value > > >::manager(boost::detail::function::function_buffer const&, boost::detail::function::function_buffer&, boost::detail::function::functor_manager_operation_type, boost::detail::function::function_obj_tag) /usr/include/boost/function/function_base.hpp:412 #3 0x20d1db8 in boost::detail::function::functor_manager, boost::_bi::list2, boost::_bi::value > > >::manage(boost::detail::function::function_buffer const&, boost::detail::function::function_buffer&, boost::detail::function::functor_manager_operation_type) /usr/include/boost/function/function_base.hpp:440 #4 0x376403f in boost::function0::assign_to_own(boost::function0 const&) /usr/include/boost/function/function_template.hpp:909 #5 0x3763cc5 in function0 /usr/include/boost/function/function_template.hpp:749 #6 0x3762e9d in function /usr/include/boost/function/function_template.hpp:1089 #7 0x3b96cf6 in ResettableHeartbeaterThread /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:82 #8 0x3b94256 in ResettableHeartbeater /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:57 (discriminator 1) #9 0x2091e4d in Peer /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:246 (discriminator 9) #10 0x2090674 in kudu::consensus::Peer::NewRemotePeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, gscoped_ptr >, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:221 (discriminator 1) #11 0x209ea8e in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:443 (discriminator 4) #12 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #13 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #14 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #15 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #16 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #17 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #18 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #19 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #20 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #21 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #22 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #23 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #24 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #25 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #26 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #27 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #28 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #29 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 Indirect leak of 57 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x148b62f in kudu::tserver::TSTabletManager::CreateNewTablet(std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::Schema const&, kudu::metadata::QuorumPB, scoped_refptr*) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:234 #3 0x1419375 in kudu::tserver::TabletServiceAdminImpl::CreateTablet(kudu::tserver::CreateTabletRequestPB const*, kudu::tserver::CreateTabletResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:281 (discriminator 2) #4 0x3329e95 in kudu::tserver::TabletServerAdminServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/tserver/tserver_admin.service.cc:48 (discriminator 1) #5 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #6 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #7 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #8 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #9 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #10 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #11 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #12 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 57 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x21f227c in RaftConsensus /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:64 #3 0x18b1787 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:134 #4 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #5 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #6 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #7 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #8 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #9 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #10 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #11 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #12 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #13 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #14 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #15 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #16 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #17 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #18 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 57 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #3 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #4 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #5 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #6 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #7 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #8 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #9 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #10 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #11 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #12 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #13 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #14 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #15 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #16 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 48 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21ba073 in __gnu_cxx::new_allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*>, false> >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x21b64b7 in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::LeaderElection::VoterState*>, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> >, std::_Select1st, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_allocate_node(std::pair, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:429 #3 0x21b587f in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::LeaderElection::VoterState*>, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> >, std::_Select1st, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_insert_bucket(std::pair, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> const&, unsigned long, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:854 #4 0x21b4b7f in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::LeaderElection::VoterState*>, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> >, std::_Select1st, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_insert(std::pair, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> const&, std::tr1::integral_constant) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:897 #5 0x21b4183 in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::LeaderElection::VoterState*>, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> >, std::_Select1st, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::insert(std::pair, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:375 #6 0x21b3c3f in bool InsertIfNotPresent, std::allocator >, kudu::consensus::LeaderElection::VoterState*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> > > >(std::tr1::unordered_map, std::allocator >, kudu::consensus::LeaderElection::VoterState*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> > >*, std::tr1::unordered_map, std::allocator >, kudu::consensus::LeaderElection::VoterState*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> > >::value_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:344 #7 0x21b38a7 in bool InsertIfNotPresent, std::allocator >, kudu::consensus::LeaderElection::VoterState*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> > > >(std::tr1::unordered_map, std::allocator >, kudu::consensus::LeaderElection::VoterState*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> > >*, std::tr1::unordered_map, std::allocator >, kudu::consensus::LeaderElection::VoterState*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> > >::value_type::first_type const&, std::tr1::unordered_map, std::allocator >, kudu::consensus::LeaderElection::VoterState*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:353 #8 0x219d595 in void InsertOrDie, std::allocator >, kudu::consensus::LeaderElection::VoterState*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> > > >(std::tr1::unordered_map, std::allocator >, kudu::consensus::LeaderElection::VoterState*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> > >*, std::tr1::unordered_map, std::allocator >, kudu::consensus::LeaderElection::VoterState*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> > >::value_type::first_type const&, std::tr1::unordered_map, std::allocator >, kudu::consensus::LeaderElection::VoterState*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::LeaderElection::VoterState*> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:377 #9 0x218aa88 in LeaderElection /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:152 #10 0x21fe041 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:192 #11 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #12 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #13 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #14 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #15 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #16 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #17 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #18 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #19 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #20 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 48 byte(s) in 3 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0xe497d6 in __gnu_cxx::new_allocator, std::allocator >, false> >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0xe44d07 in std::tr1::_Hashtable, std::allocator >, std::basic_string, std::allocator >, std::allocator, std::allocator > >, std::_Identity, std::allocator > >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, true, true>::_M_allocate_node(std::basic_string, std::allocator > const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:429 #3 0x2254678 in _Hashtable /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:602 #4 0x2253dcd in __unordered_set /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/unordered_set.h:43 #5 0x2253bdd in unordered_set /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/unordered_set.h:173 #6 0x2257c2f in QuorumState /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus_state.cc:86 #7 0x225712e in kudu::consensus::QuorumState::Build(kudu::metadata::QuorumPB const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus_state.cc:70 #8 0x2259fce in kudu::consensus::ReplicaState::ResetActiveQuorumStateUnlocked(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus_state.cc:659 #9 0x2262b01 in kudu::consensus::ReplicaState::SetPendingQuorumUnlocked(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus_state.cc:260 #10 0x2202c96 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1024 (discriminator 2) #11 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #12 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #13 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #14 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #15 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #16 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #17 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #18 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #19 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #20 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #21 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #22 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #23 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #24 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #25 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #26 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #27 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #28 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #29 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) Indirect leak of 48 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21cdfe3 in __gnu_cxx::new_allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x21cca0f in std::_Rb_tree, std::allocator >, std::pair, std::allocator > const, kudu::consensus::ElectionVote>, std::_Select1st, std::allocator > const, kudu::consensus::ElectionVote> >, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::_M_get_node() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_tree.h:370 #3 0x21cc3dc in std::_Rb_tree, std::allocator >, std::pair, std::allocator > const, kudu::consensus::ElectionVote>, std::_Select1st, std::allocator > const, kudu::consensus::ElectionVote> >, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::_M_create_node(std::pair, std::allocator > const, kudu::consensus::ElectionVote> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_tree.h:380 #4 0x21cb775 in std::_Rb_tree, std::allocator >, std::pair, std::allocator > const, kudu::consensus::ElectionVote>, std::_Select1st, std::allocator > const, kudu::consensus::ElectionVote> >, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::_M_insert_(std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::pair, std::allocator > const, kudu::consensus::ElectionVote> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_tree.h:1023 #5 0x21ca323 in std::_Rb_tree, std::allocator >, std::pair, std::allocator > const, kudu::consensus::ElectionVote>, std::_Select1st, std::allocator > const, kudu::consensus::ElectionVote> >, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::_M_insert_unique(std::pair, std::allocator > const, kudu::consensus::ElectionVote> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_tree.h:1380 #6 0x21c9b89 in std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::insert(std::pair, std::allocator > const, kudu::consensus::ElectionVote> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_map.h:595 #7 0x21c934e in bool InsertIfNotPresent, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > > >(std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >*, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:344 #8 0x21c8fa7 in bool InsertIfNotPresent, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > > >(std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >*, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type::first_type const&, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:353 #9 0x2199765 in void InsertOrDie, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > > >(std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >*, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type::first_type const&, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:377 #10 0x218682c in kudu::consensus::VoteCounter::RegisterVote(std::basic_string, std::allocator > const&, kudu::consensus::ElectionVote, bool*) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:76 #11 0x21fd1b3 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:181 (discriminator 4) #12 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #13 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #14 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #15 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #16 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #17 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #18 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #19 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #20 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #21 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 48 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20a555f in kudu::consensus::RpcPeerProxyFactory::NewProxy(kudu::metadata::QuorumPeerPB const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:542 #2 0x209e19c in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:439 (discriminator 1) #3 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #4 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #5 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #6 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #7 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #8 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #9 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #10 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #11 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #12 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #13 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #14 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #15 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #16 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #17 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #18 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #19 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #20 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #21 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #22 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #23 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #24 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #25 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 48 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20c9093 in __gnu_cxx::new_allocator, std::allocator > const, kudu::consensus::Peer*>, false> >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x20c4f17 in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::Peer*>, std::allocator, std::allocator > const, kudu::consensus::Peer*> >, std::_Select1st, std::allocator > const, kudu::consensus::Peer*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_allocate_node(std::pair, std::allocator > const, kudu::consensus::Peer*> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:429 #3 0x20c42df in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::Peer*>, std::allocator, std::allocator > const, kudu::consensus::Peer*> >, std::_Select1st, std::allocator > const, kudu::consensus::Peer*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_insert_bucket(std::pair, std::allocator > const, kudu::consensus::Peer*> const&, unsigned long, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:854 #4 0x20c29cf in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::Peer*>, std::allocator, std::allocator > const, kudu::consensus::Peer*> >, std::_Select1st, std::allocator > const, kudu::consensus::Peer*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_insert(std::pair, std::allocator > const, kudu::consensus::Peer*> const&, std::tr1::integral_constant) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:897 #5 0x20c1fd3 in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::Peer*>, std::allocator, std::allocator > const, kudu::consensus::Peer*> >, std::_Select1st, std::allocator > const, kudu::consensus::Peer*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::insert(std::pair, std::allocator > const, kudu::consensus::Peer*> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:375 #6 0x20c1a8f in bool InsertIfNotPresent, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > > >(std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >*, std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >::value_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:344 #7 0x20c16f7 in bool InsertIfNotPresent, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > > >(std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >*, std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >::value_type::first_type const&, std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:353 #8 0x20b5675 in void InsertOrDie, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > > >(std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >*, std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >::value_type::first_type const&, std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:377 #9 0x209f0d5 in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:449 #10 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #11 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #12 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #13 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #14 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #15 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #16 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #17 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #18 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #19 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #20 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #21 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #22 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #23 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #24 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #25 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #26 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #27 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #28 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #29 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) Indirect leak of 48 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21cdfe3 in __gnu_cxx::new_allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x21cca0f in std::_Rb_tree, std::allocator >, std::pair, std::allocator > const, kudu::consensus::ElectionVote>, std::_Select1st, std::allocator > const, kudu::consensus::ElectionVote> >, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::_M_get_node() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_tree.h:370 #3 0x21cc3dc in std::_Rb_tree, std::allocator >, std::pair, std::allocator > const, kudu::consensus::ElectionVote>, std::_Select1st, std::allocator > const, kudu::consensus::ElectionVote> >, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::_M_create_node(std::pair, std::allocator > const, kudu::consensus::ElectionVote> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_tree.h:380 #4 0x21cb775 in std::_Rb_tree, std::allocator >, std::pair, std::allocator > const, kudu::consensus::ElectionVote>, std::_Select1st, std::allocator > const, kudu::consensus::ElectionVote> >, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::_M_insert_(std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::pair, std::allocator > const, kudu::consensus::ElectionVote> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_tree.h:1023 #5 0x21ca323 in std::_Rb_tree, std::allocator >, std::pair, std::allocator > const, kudu::consensus::ElectionVote>, std::_Select1st, std::allocator > const, kudu::consensus::ElectionVote> >, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::_M_insert_unique(std::pair, std::allocator > const, kudu::consensus::ElectionVote> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_tree.h:1380 #6 0x21c9b89 in std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::insert(std::pair, std::allocator > const, kudu::consensus::ElectionVote> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_map.h:595 #7 0x21c934e in bool InsertIfNotPresent, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > > >(std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >*, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:344 #8 0x21c8fa7 in bool InsertIfNotPresent, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > > >(std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >*, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type::first_type const&, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:353 #9 0x2199765 in void InsertOrDie, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > > >(std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >*, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type::first_type const&, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:377 #10 0x218682c in kudu::consensus::VoteCounter::RegisterVote(std::basic_string, std::allocator > const&, kudu::consensus::ElectionVote, bool*) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:76 #11 0x2191499 in kudu::consensus::LeaderElection::RecordVoteUnlocked(std::basic_string, std::allocator > const&, kudu::consensus::ElectionVote) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:255 #12 0x218f4c7 in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:220 #13 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #14 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #15 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #16 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #17 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #18 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #19 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #20 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #21 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #22 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #23 0x3694516 in kudu::rpc::OutboundCall::SetFailed(kudu::Status const&, kudu::rpc::ErrorStatusPB*) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:203 #24 0x37ad393 in kudu::rpc::Connection::Shutdown(kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:124 #25 0x36fda77 in kudu::rpc::ReactorThread::DestroyConnection(kudu::rpc::Connection*, kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/rpc/reactor.cc:414 #26 0x3709d0e in kudu::rpc::ReactorThread::CompleteConnectionNegotiation(scoped_refptr const&, kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/rpc/reactor.cc:355 #27 0x37ff050 in kudu::rpc::NegotiationCompletedTask::Run(kudu::rpc::ReactorThread*) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:544 #28 0x36fb25a in kudu::rpc::ReactorThread::AsyncHandler(ev::async&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/reactor.cc:172 #29 0x375e2ae in void ev::base::method_thunk(ev_loop*, ev_async*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 12) Indirect leak of 48 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21cdfe3 in __gnu_cxx::new_allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x21cca0f in std::_Rb_tree, std::allocator >, std::pair, std::allocator > const, kudu::consensus::ElectionVote>, std::_Select1st, std::allocator > const, kudu::consensus::ElectionVote> >, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::_M_get_node() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_tree.h:370 #3 0x21cc3dc in std::_Rb_tree, std::allocator >, std::pair, std::allocator > const, kudu::consensus::ElectionVote>, std::_Select1st, std::allocator > const, kudu::consensus::ElectionVote> >, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::_M_create_node(std::pair, std::allocator > const, kudu::consensus::ElectionVote> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_tree.h:380 #4 0x21cb775 in std::_Rb_tree, std::allocator >, std::pair, std::allocator > const, kudu::consensus::ElectionVote>, std::_Select1st, std::allocator > const, kudu::consensus::ElectionVote> >, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::_M_insert_(std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::pair, std::allocator > const, kudu::consensus::ElectionVote> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_tree.h:1023 #5 0x21ca323 in std::_Rb_tree, std::allocator >, std::pair, std::allocator > const, kudu::consensus::ElectionVote>, std::_Select1st, std::allocator > const, kudu::consensus::ElectionVote> >, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::_M_insert_unique(std::pair, std::allocator > const, kudu::consensus::ElectionVote> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_tree.h:1380 #6 0x21c9b89 in std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::insert(std::pair, std::allocator > const, kudu::consensus::ElectionVote> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_map.h:595 #7 0x21c934e in bool InsertIfNotPresent, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > > >(std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >*, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:344 #8 0x21c8fa7 in bool InsertIfNotPresent, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > > >(std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >*, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type::first_type const&, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:353 #9 0x2199765 in void InsertOrDie, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > > >(std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >*, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type::first_type const&, std::map, std::allocator >, kudu::consensus::ElectionVote, std::less, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::ElectionVote> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:377 #10 0x218682c in kudu::consensus::VoteCounter::RegisterVote(std::basic_string, std::allocator > const&, kudu::consensus::ElectionVote, bool*) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:76 #11 0x2191499 in kudu::consensus::LeaderElection::RecordVoteUnlocked(std::basic_string, std::allocator > const&, kudu::consensus::ElectionVote) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:255 #12 0x2193957 in kudu::consensus::LeaderElection::HandleVoteGrantedUnlocked(std::basic_string, std::allocator > const&, kudu::consensus::LeaderElection::VoterState const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:301 #13 0x2190427 in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:238 #14 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #15 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #16 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #17 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #18 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #19 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #20 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #21 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #22 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #23 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #24 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #25 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #26 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #27 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #28 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 48 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20a555f in kudu::consensus::RpcPeerProxyFactory::NewProxy(kudu::metadata::QuorumPeerPB const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:542 #2 0x21f656a in kudu::consensus::RaftConsensus::Start(kudu::consensus::ConsensusBootstrapInfo const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:111 (discriminator 2) #3 0x18b4e50 in kudu::tablet::TabletPeer::Start(kudu::consensus::ConsensusBootstrapInfo const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:176 (discriminator 1) #4 0x1486d90 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:364 (discriminator 1) #5 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #6 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #7 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #8 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #9 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #10 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #11 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #12 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #13 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #14 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #15 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #16 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #17 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #18 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 48 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21c4633 in __gnu_cxx::new_allocator, std::allocator > const, kudu::consensus::PeerProxy*>, false> >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x21c3cc7 in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::PeerProxy*>, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> >, std::_Select1st, std::allocator > const, kudu::consensus::PeerProxy*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_allocate_node(std::pair, std::allocator > const, kudu::consensus::PeerProxy*> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:429 #3 0x22469df in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::PeerProxy*>, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> >, std::_Select1st, std::allocator > const, kudu::consensus::PeerProxy*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_insert_bucket(std::pair, std::allocator > const, kudu::consensus::PeerProxy*> const&, unsigned long, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:854 #4 0x22450cf in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::PeerProxy*>, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> >, std::_Select1st, std::allocator > const, kudu::consensus::PeerProxy*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_insert(std::pair, std::allocator > const, kudu::consensus::PeerProxy*> const&, std::tr1::integral_constant) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:897 #5 0x22446d3 in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::PeerProxy*>, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> >, std::_Select1st, std::allocator > const, kudu::consensus::PeerProxy*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::insert(std::pair, std::allocator > const, kudu::consensus::PeerProxy*> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:375 #6 0x224418f in bool InsertIfNotPresent, std::allocator >, kudu::consensus::PeerProxy*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> > > >(std::tr1::unordered_map, std::allocator >, kudu::consensus::PeerProxy*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> > >*, std::tr1::unordered_map, std::allocator >, kudu::consensus::PeerProxy*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> > >::value_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:344 #7 0x2243df7 in bool InsertIfNotPresent, std::allocator >, kudu::consensus::PeerProxy*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> > > >(std::tr1::unordered_map, std::allocator >, kudu::consensus::PeerProxy*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> > >*, std::tr1::unordered_map, std::allocator >, kudu::consensus::PeerProxy*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> > >::value_type::first_type const&, std::tr1::unordered_map, std::allocator >, kudu::consensus::PeerProxy*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:353 #8 0x2239b95 in void InsertOrDie, std::allocator >, kudu::consensus::PeerProxy*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> > > >(std::tr1::unordered_map, std::allocator >, kudu::consensus::PeerProxy*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> > >*, std::tr1::unordered_map, std::allocator >, kudu::consensus::PeerProxy*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> > >::value_type::first_type const&, std::tr1::unordered_map, std::allocator >, kudu::consensus::PeerProxy*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::PeerProxy*> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:377 #9 0x21f69f1 in kudu::consensus::RaftConsensus::Start(kudu::consensus::ConsensusBootstrapInfo const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:112 #10 0x18b4e50 in kudu::tablet::TabletPeer::Start(kudu::consensus::ConsensusBootstrapInfo const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:176 (discriminator 1) #11 0x1486d90 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:364 (discriminator 1) #12 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #13 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #14 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #15 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #16 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #17 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #18 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #19 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #20 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #21 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #22 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #23 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #24 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #25 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 47 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x7f7f2ca1bef7 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) ??:? #3 0x7f7ef189155f () #4 0x7f7ef189236f () Indirect leak of 45 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x7f7f2ca1bef7 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) ??:? #3 0x6030003ec2ff () Indirect leak of 45 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x7f7f2ca1bef7 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) ??:? #3 0x6030003ec26f () Indirect leak of 41 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x7f7f2ca1bef7 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) ??:? #3 0x61900038e57f () #3 0x3f62c84 in ?? ??:0 Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x223dd56 in kudu::consensus::ConsensusStatusPB::mutable_error() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:3713 (discriminator 1) #2 0x3258ba9 in kudu::consensus::ConsensusStatusPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:3949 #3 0x32d0137 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::consensus::ConsensusStatusPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #4 0x3287a60 in kudu::consensus::ConsensusResponsePB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:5448 #5 0x3eaa5ca in google::protobuf::(anonymous namespace)::InlineMergeFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/message_lite.cc:121 #6 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #7 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #8 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #9 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x223e436 in kudu::consensus::ConsensusErrorPB::mutable_status() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:2537 (discriminator 1) #2 0x31fed4e in kudu::consensus::ConsensusErrorPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:910 #3 0x32ce4c7 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::consensus::ConsensusErrorPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #4 0x3258bb6 in kudu::consensus::ConsensusStatusPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:3949 #5 0x32d0137 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::consensus::ConsensusStatusPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #6 0x3287a60 in kudu::consensus::ConsensusResponsePB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:5448 #7 0x3eaa5ca in google::protobuf::(anonymous namespace)::InlineMergeFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/message_lite.cc:121 #8 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #9 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #10 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #11 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3a8248f in kudu::(anonymous namespace)::InitiateHighWaterMark(std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, gscoped_ptr, kudu::DefaultDeleter > >*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:39 #2 0x3a819aa in MemTracker /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:74 #3 0x3a80131 in kudu::MemTracker::CreateTracker(long, std::basic_string, std::allocator > const&, kudu::MemTracker*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:47 #4 0x2102860 in PeerMessageQueue /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:116 #5 0x18b103a in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 (discriminator 1) #6 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #7 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #8 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #9 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #10 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #11 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #12 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #13 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #14 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #15 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #16 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #17 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #18 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #19 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #20 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3a8248f in kudu::(anonymous namespace)::InitiateHighWaterMark(std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, gscoped_ptr, kudu::DefaultDeleter > >*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:39 #2 0x3a819aa in MemTracker /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:74 #3 0x3a87854 in kudu::MemTracker::FindOrCreateTracker(long, std::basic_string, std::allocator > const&, kudu::MemTracker*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:136 #4 0x21024b7 in PeerMessageQueue /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:112 #5 0x18b103a in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 (discriminator 1) #6 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #7 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #8 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #9 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #10 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #11 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #12 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #13 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #14 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #15 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #16 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #17 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #18 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #19 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #20 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x223bea5 in _ZN4kudu4BindIMNS_9consensus13RaftConsensusEFvRKNS1_14ElectionResultEEPS2_EENS_8CallbackINS_8internal9BindStateINSA_13FunctorTraitsIT_E12RunnableTypeENSE_7RunTypeEFvNSA_19CallbackParamTraitsIT0_Xsr14IsMoveOnlyTypeISI_EE5valueEE11StorageTypeEEE14UnboundRunTypeEEESD_RKSI_ /srv/home/alex/dev/kudu/src/kudu/gutil/bind.h:118 #2 0x21fe00f in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:193 #3 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #4 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #5 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #6 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #7 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #8 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #9 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #10 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #11 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #12 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x223b7e6 in kudu::consensus::VoteRequestPB::mutable_candidate_status() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:3917 (discriminator 1) #2 0x325dd63 in kudu::consensus::VoteRequestPB::MergeFrom(kudu::consensus::VoteRequestPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:4429 #3 0x325d3c4 in VoteRequestPB /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:4136 #4 0x2189514 in LeaderElection /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:147 (discriminator 2) #5 0x21fe041 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:192 #6 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #7 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #8 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #9 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #10 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #11 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #12 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #13 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #14 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #15 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20fe386 in kudu::consensus::ConsensusStatusPB::mutable_last_received() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:3675 (discriminator 1) #2 0x3256275 in kudu::consensus::ConsensusStatusPB::MergeFrom(kudu::consensus::ConsensusStatusPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:4063 #3 0x325dda2 in kudu::consensus::VoteRequestPB::MergeFrom(kudu::consensus::VoteRequestPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:4429 #4 0x325d3c4 in VoteRequestPB /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:4136 #5 0x2189514 in LeaderElection /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:147 (discriminator 2) #6 0x21fe041 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:192 #7 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #8 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #9 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #10 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #11 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #12 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #13 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #14 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #15 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #16 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x36df8c0 in __shared_count > /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:317 #2 0x36df389 in __shared_ptr > /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:563 #3 0x36b9e51 in shared_ptr > /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:999 #4 0x36aa4da in kudu::rpc::MessengerBuilder::Build(std::tr1::shared_ptr*) /srv/home/alex/dev/kudu/src/kudu/rpc/messenger.cc:93 #5 0x15fcf6f in kudu::server::ServerBase::Init() /srv/home/alex/dev/kudu/src/kudu/server/server_base.cc:119 (discriminator 1) #6 0x1405c6f in kudu::tserver::TabletServer::Init() /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_server.cc:76 (discriminator 1) #7 0x13ba717 in kudu::tserver::MiniTabletServer::Start() /srv/home/alex/dev/kudu/src/kudu/tserver/mini_tablet_server.cc:56 (discriminator 2) #8 0x1023e51 in kudu::MiniCluster::AddTabletServer() /srv/home/alex/dev/kudu/src/kudu/integration-tests/mini_cluster.cc:179 (discriminator 2) #9 0x101c840 in kudu::MiniCluster::Start() /srv/home/alex/dev/kudu/src/kudu/integration-tests/mini_cluster.cc:78 (discriminator 1) #10 0xca88d4 in kudu::client::ClientTest::SetUp() /srv/home/alex/dev/kudu/src/kudu/client/client-test.cc:95 (discriminator 1) #11 0x3f4b702 in void testing::internal::HandleSehExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2078 #12 0x3f3ed70 in void testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2114 #13 0x3f25432 in testing::Test::Run() /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2146 #14 0x3f262ca in testing::TestInfo::Run() /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2326 #15 0x3f26af6 in testing::TestCase::Run() /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2444 #16 0x3f2ebc6 in testing::internal::UnitTestImpl::RunAllTests() /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:4315 #17 0x3f48a22 in bool testing::internal::HandleSehExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2078 #18 0x3f41100 in bool testing::internal::HandleExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:2114 #19 0x3f2e84a in testing::UnitTest::Run() /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/src/gtest.cc:3926 #20 0x1065db0 in RUN_ALL_TESTS() /srv/home/alex/dev/kudu/thirdparty/gmock-1.7.0/gtest/include/gtest/gtest.h:2288 #21 0x1064211 in main /srv/home/alex/dev/kudu/src/kudu/util/test_main.cc:34 #22 0x7f7f2c3a5ec4 in __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:287 Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x7f7f2ca1bef7 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) ??:? #3 0x603000415f3f () Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x1141a16 in kudu::metadata::QuorumPeerPB::mutable_last_known_addr() /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.h:426 (discriminator 1) #2 0x35b07ae in kudu::metadata::QuorumPeerPB::MergeFrom(kudu::metadata::QuorumPeerPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:431 #3 0x35afe84 in QuorumPeerPB /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:187 #4 0x2091895 in Peer /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:246 (discriminator 1) #5 0x208de9f in kudu::consensus::Peer::NewLocalPeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, kudu::log::Log*, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:203 (discriminator 1) #6 0x209d050 in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:429 (discriminator 1) #7 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #8 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #9 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #10 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #11 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #12 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #13 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #14 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #15 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #16 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #17 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #18 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #19 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #20 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #21 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #22 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #23 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #24 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #25 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #26 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #27 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #28 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #29 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20fdfc6 in kudu::consensus::ConsensusResponsePB::mutable_status() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:4512 (discriminator 1) #2 0x20e646c in kudu::consensus::LocalPeer::SetupResponse(kudu::consensus::ConsensusResponsePB*, unsigned long) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:123 #3 0x20e354e in kudu::consensus::LocalPeer::SendRequest(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::Callback const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:89 #4 0x209413e in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:311 #5 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #6 0x20a0d87 in kudu::consensus::PeerManager::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:467 (discriminator 1) #7 0x220b6bb in kudu::consensus::RaftConsensus::Replicate(kudu::consensus::ConsensusRound*) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:303 (discriminator 1) #8 0x191990d in kudu::tablet::TransactionDriver::PrepareAndStart() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:159 #9 0x1917012 in kudu::tablet::TransactionDriver::PrepareAndStartTask() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:114 #10 0x192ad39 in kudu::internal::RunnableAdapter::Run(kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:134 (discriminator 4) #11 0x192a908 in kudu::internal::InvokeHelper, void (kudu::tablet::TransactionDriver*)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:871 #12 0x192a58c in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::tablet::TransactionDriver*), void (kudu::internal::UnretainedWrapper)>, void (kudu::tablet::TransactionDriver*)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1063 #13 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #14 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #15 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #16 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #17 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #18 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #19 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #20 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #21 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #22 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #23 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #24 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #25 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #26 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #27 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20fe386 in kudu::consensus::ConsensusStatusPB::mutable_last_received() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:3675 (discriminator 1) #2 0x20e64a6 in kudu::consensus::LocalPeer::SetupResponse(kudu::consensus::ConsensusResponsePB*, unsigned long) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:124 #3 0x20e354e in kudu::consensus::LocalPeer::SendRequest(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::Callback const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:89 #4 0x209413e in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:311 #5 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #6 0x20a0d87 in kudu::consensus::PeerManager::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:467 (discriminator 1) #7 0x220b6bb in kudu::consensus::RaftConsensus::Replicate(kudu::consensus::ConsensusRound*) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:303 (discriminator 1) #8 0x191990d in kudu::tablet::TransactionDriver::PrepareAndStart() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:159 #9 0x1917012 in kudu::tablet::TransactionDriver::PrepareAndStartTask() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:114 #10 0x192ad39 in kudu::internal::RunnableAdapter::Run(kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:134 (discriminator 4) #11 0x192a908 in kudu::internal::InvokeHelper, void (kudu::tablet::TransactionDriver*)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:871 #12 0x192a58c in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::tablet::TransactionDriver*), void (kudu::internal::UnretainedWrapper)>, void (kudu::tablet::TransactionDriver*)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1063 #13 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #14 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #15 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #16 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #17 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #18 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #19 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #20 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #21 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #22 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #23 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #24 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #25 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #26 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #27 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x7f7f2ca1bef7 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) ??:? #3 0x603000415cff () Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20fdfc6 in kudu::consensus::ConsensusResponsePB::mutable_status() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:4512 (discriminator 1) #2 0x3287a50 in kudu::consensus::ConsensusResponsePB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:5448 #3 0x3eaa5ca in google::protobuf::(anonymous namespace)::InlineMergeFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/message_lite.cc:121 #4 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #5 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #6 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #7 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 40 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20fe386 in kudu::consensus::ConsensusStatusPB::mutable_last_received() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:3675 (discriminator 1) #2 0x3258279 in kudu::consensus::ConsensusStatusPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:3935 #3 0x32d0137 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::consensus::ConsensusStatusPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #4 0x3287a60 in kudu::consensus::ConsensusResponsePB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:5448 #5 0x3eaa5ca in google::protobuf::(anonymous namespace)::InlineMergeFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/message_lite.cc:121 #6 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #7 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #8 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #9 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 36 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x7f7f2ca1bef7 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) ??:? #3 0xff06185ffbb () #4 0x60600040c287 () Indirect leak of 35 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x7f7f2ca1a3b8 in std::basic_string, std::allocator >::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) ??:? #2 0x7f7f2ca1bef7 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) ??:? #3 0x61400019ccc7 () Indirect leak of 32 byte(s) in 1 object(s) allocated from: #0 0xafd1fb in operator new[](unsigned long) ??:? #1 0x3eab220 in google::protobuf::internal::RepeatedPtrFieldBase::Reserve(int) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/repeated_field.cc:51 #2 0x2136c5a in google::protobuf::RepeatedPtrField::AddAllocated(kudu::consensus::ReplicateMsg*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:1127 #3 0x2115943 in kudu::consensus::PeerMessageQueue::RequestForPeer(std::basic_string, std::allocator > const&, kudu::consensus::ConsensusRequestPB*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:332 (discriminator 1) #4 0x2093607 in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:289 #5 0x2095fd7 in kudu::consensus::Peer::ProcessResponse(kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:340 #6 0x20cfc13 in kudu::internal::RunnableAdapter::Run(kudu::consensus::Peer*, kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #7 0x20cf752 in kudu::internal::InvokeHelper, void (kudu::consensus::Peer*, kudu::Status const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::Peer*, kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #8 0x20cf33f in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::Peer*, kudu::Status const&), void (kudu::internal::UnretainedWrapper)>, void (kudu::consensus::Peer*, kudu::Status const&)>::Run(kudu::internal::BindStateBase*, kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #9 0xe18b8e in kudu::Callback::Run(kudu::Status const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #10 0x20e35ac in kudu::consensus::LocalPeer::SendRequest(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::Callback const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:90 #11 0x209413e in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:311 #12 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #13 0x20a0d87 in kudu::consensus::PeerManager::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:467 (discriminator 1) #14 0x220b6bb in kudu::consensus::RaftConsensus::Replicate(kudu::consensus::ConsensusRound*) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:303 (discriminator 1) #15 0x191990d in kudu::tablet::TransactionDriver::PrepareAndStart() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:159 #16 0x1917012 in kudu::tablet::TransactionDriver::PrepareAndStartTask() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:114 #17 0x192ad39 in kudu::internal::RunnableAdapter::Run(kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:134 (discriminator 4) #18 0x192a908 in kudu::internal::InvokeHelper, void (kudu::tablet::TransactionDriver*)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:871 #19 0x192a58c in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::tablet::TransactionDriver*), void (kudu::internal::UnretainedWrapper)>, void (kudu::tablet::TransactionDriver*)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1063 #20 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #21 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #22 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #23 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #24 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #25 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #26 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #27 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #28 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #29 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 Indirect leak of 32 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20a4afa in kudu::consensus::RpcPeerProxyFactory::NewProxy(kudu::metadata::QuorumPeerPB const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:538 #2 0x21f656a in kudu::consensus::RaftConsensus::Start(kudu::consensus::ConsensusBootstrapInfo const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:111 (discriminator 2) #3 0x18b4e50 in kudu::tablet::TabletPeer::Start(kudu::consensus::ConsensusBootstrapInfo const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:176 (discriminator 1) #4 0x1486d90 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:364 (discriminator 1) #5 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #6 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #7 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #8 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #9 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #10 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #11 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #12 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #13 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #14 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #15 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #16 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #17 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #18 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 32 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x21925c8 in kudu::consensus::LeaderElection::RecordVoteUnlocked(std::basic_string, std::allocator > const&, kudu::consensus::ElectionVote) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:275 #2 0x2193957 in kudu::consensus::LeaderElection::HandleVoteGrantedUnlocked(std::basic_string, std::allocator > const&, kudu::consensus::LeaderElection::VoterState const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:301 #3 0x2190427 in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:238 #4 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #5 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #6 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #7 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #8 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #9 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #10 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #11 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #12 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #13 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #14 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #15 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #16 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #17 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #18 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 32 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20d4802 in boost::detail::function::functor_manager, boost::_bi::list2, boost::_bi::value > > >::manager(boost::detail::function::function_buffer const&, boost::detail::function::function_buffer&, boost::detail::function::functor_manager_operation_type, mpl_::bool_) /usr/include/boost/function/function_base.hpp:380 #2 0x20d441a in boost::detail::function::functor_manager, boost::_bi::list2, boost::_bi::value > > >::manager(boost::detail::function::function_buffer const&, boost::detail::function::function_buffer&, boost::detail::function::functor_manager_operation_type, boost::detail::function::function_obj_tag) /usr/include/boost/function/function_base.hpp:412 #3 0x20d1db8 in boost::detail::function::functor_manager, boost::_bi::list2, boost::_bi::value > > >::manage(boost::detail::function::function_buffer const&, boost::detail::function::function_buffer&, boost::detail::function::functor_manager_operation_type) /usr/include/boost/function/function_base.hpp:440 #4 0x376403f in boost::function0::assign_to_own(boost::function0 const&) /usr/include/boost/function/function_template.hpp:909 #5 0x3763cc5 in function0 /usr/include/boost/function/function_template.hpp:749 #6 0x3762e9d in function /usr/include/boost/function/function_template.hpp:1089 #7 0x3b96cf6 in ResettableHeartbeaterThread /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:82 #8 0x3b94256 in ResettableHeartbeater /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:57 (discriminator 1) #9 0x2091e4d in Peer /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:246 (discriminator 9) #10 0x208de9f in kudu::consensus::Peer::NewLocalPeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, kudu::log::Log*, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:203 (discriminator 1) #11 0x209d050 in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:429 (discriminator 1) #12 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #13 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #14 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #15 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #16 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #17 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #18 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #19 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #20 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #21 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #22 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #23 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #24 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #25 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #26 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #27 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #28 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #29 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 Indirect leak of 32 byte(s) in 1 object(s) allocated from: #0 0xafd1fb in operator new[](unsigned long) ??:? #1 0x3eab220 in google::protobuf::internal::RepeatedPtrFieldBase::Reserve(int) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/repeated_field.cc:51 #2 0x2136c5a in google::protobuf::RepeatedPtrField::AddAllocated(kudu::consensus::ReplicateMsg*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:1127 #3 0x2115943 in kudu::consensus::PeerMessageQueue::RequestForPeer(std::basic_string, std::allocator > const&, kudu::consensus::ConsensusRequestPB*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:332 (discriminator 1) #4 0x2093607 in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:289 #5 0x2095fd7 in kudu::consensus::Peer::ProcessResponse(kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:340 #6 0x20cfc13 in kudu::internal::RunnableAdapter::Run(kudu::consensus::Peer*, kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #7 0x20cf752 in kudu::internal::InvokeHelper, void (kudu::consensus::Peer*, kudu::Status const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::Peer*, kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #8 0x20cf33f in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::Peer*, kudu::Status const&), void (kudu::internal::UnretainedWrapper)>, void (kudu::consensus::Peer*, kudu::Status const&)>::Run(kudu::internal::BindStateBase*, kudu::Status const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #9 0xe18b8e in kudu::Callback::Run(kudu::Status const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #10 0x20db4c5 in kudu::consensus::RemotePeer::RequestCallback(kudu::Callback const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:188 #11 0x20deebb in boost::_mfi::mf1 const&>::operator()(kudu::consensus::RemotePeer*, kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 8) #12 0x20de989 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #13 0x20de60a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x20dd091 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #17 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #18 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #19 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #20 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #21 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 32 byte(s) in 1 object(s) allocated from: #0 0xafd1fb in operator new[](unsigned long) ??:? #1 0x3eab220 in google::protobuf::internal::RepeatedPtrFieldBase::Reserve(int) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/repeated_field.cc:51 #2 0x127132b in google::protobuf::RepeatedPtrField::Add() /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/repeated_field.h:1048 #3 0x1140ef2 in kudu::metadata::QuorumPB::add_peers() /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.h:507 #4 0x35bce00 in kudu::metadata::QuorumPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:588 #5 0x3117df7 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::metadata::QuorumPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #6 0x320d441 in kudu::consensus::ConsensusMetadataPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:1411 #7 0x3eaa5ca in google::protobuf::(anonymous namespace)::InlineMergeFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/message_lite.cc:121 #8 0x208b3c0 in kudu::consensus::ConsensusMetadata::Load(kudu::FsManager*, std::basic_string, std::allocator > const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_meta.cc:41 (discriminator 5) #9 0x18b0503 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:108 (discriminator 3) #10 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #11 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #12 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #13 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #17 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #18 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #19 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #20 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #21 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #22 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #23 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #24 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 32 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3ab743d in __shared_count /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:302 #2 0x3ab7029 in __shared_ptr /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:554 #3 0x3a9121a in shared_ptr /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:995 #4 0x3a8786b in kudu::MemTracker::FindOrCreateTracker(long, std::basic_string, std::allocator > const&, kudu::MemTracker*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:136 (discriminator 2) #5 0x21024b7 in PeerMessageQueue /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:112 #6 0x18b103a in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 (discriminator 1) #7 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #8 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #9 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #10 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #11 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #12 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #13 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #14 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #15 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #16 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #17 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #18 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #19 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #20 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #21 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 32 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3ab743d in __shared_count /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:302 #2 0x3ab7029 in __shared_ptr /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:554 #3 0x3a9121a in shared_ptr /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/shared_ptr.h:995 #4 0x3a80148 in kudu::MemTracker::CreateTracker(long, std::basic_string, std::allocator > const&, kudu::MemTracker*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:47 (discriminator 1) #5 0x2102860 in PeerMessageQueue /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:116 #6 0x18b103a in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 (discriminator 1) #7 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #8 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #9 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #10 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #11 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #12 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #13 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #14 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #15 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #16 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #17 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #18 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #19 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #20 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #21 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 32 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20a4afa in kudu::consensus::RpcPeerProxyFactory::NewProxy(kudu::metadata::QuorumPeerPB const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:538 #2 0x209e19c in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:439 (discriminator 1) #3 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #4 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #5 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #6 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #7 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #8 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #9 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #10 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #11 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #12 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #13 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #14 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #15 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #16 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #17 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #18 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #19 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #20 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #21 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #22 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #23 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #24 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #25 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 24 byte(s) in 3 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x13b8079 in kudu::HostPortPB::mutable_host() /srv/home/alex/dev/kudu/src/kudu/common/common.pb.h:938 #2 0x3669ee9 in kudu::HostPortPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/common/common.pb.cc:1143 #3 0x16c0337 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::HostPortPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #4 0x35b3cee in kudu::metadata::QuorumPeerPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:293 #5 0x35c3ce7 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::metadata::QuorumPeerPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #6 0x35bce10 in kudu::metadata::QuorumPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:588 #7 0x3117df7 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::metadata::QuorumPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #8 0x320d441 in kudu::consensus::ConsensusMetadataPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:1411 #9 0x3eaa5ca in google::protobuf::(anonymous namespace)::InlineMergeFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/message_lite.cc:121 #10 0x208b3c0 in kudu::consensus::ConsensusMetadata::Load(kudu::FsManager*, std::basic_string, std::allocator > const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_meta.cc:41 (discriminator 5) #11 0x18b0503 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:108 (discriminator 3) #12 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #13 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #14 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #15 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #16 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #17 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #18 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #19 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #20 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #21 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #22 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #23 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #24 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #25 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #26 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 24 byte(s) in 3 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20b1210 in kudu::consensus::ConsensusRequestPB::set_caller_uuid(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:4226 #2 0x209364d in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:291 #3 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #4 0x20a0d87 in kudu::consensus::PeerManager::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:467 (discriminator 1) #5 0x220b6bb in kudu::consensus::RaftConsensus::Replicate(kudu::consensus::ConsensusRound*) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:303 (discriminator 1) #6 0x191990d in kudu::tablet::TransactionDriver::PrepareAndStart() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:159 #7 0x1917012 in kudu::tablet::TransactionDriver::PrepareAndStartTask() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:114 #8 0x192ad39 in kudu::internal::RunnableAdapter::Run(kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:134 (discriminator 4) #9 0x192a908 in kudu::internal::InvokeHelper, void (kudu::tablet::TransactionDriver*)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:871 #10 0x192a58c in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::tablet::TransactionDriver*), void (kudu::internal::UnretainedWrapper)>, void (kudu::tablet::TransactionDriver*)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1063 #11 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #12 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #13 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #14 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #15 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #16 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #17 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #18 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #19 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #20 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #21 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #22 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #23 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #24 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #25 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 24 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20c9093 in __gnu_cxx::new_allocator, std::allocator > const, kudu::consensus::Peer*>, false> >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x20c4f17 in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::Peer*>, std::allocator, std::allocator > const, kudu::consensus::Peer*> >, std::_Select1st, std::allocator > const, kudu::consensus::Peer*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_allocate_node(std::pair, std::allocator > const, kudu::consensus::Peer*> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:429 #3 0x20c42df in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::Peer*>, std::allocator, std::allocator > const, kudu::consensus::Peer*> >, std::_Select1st, std::allocator > const, kudu::consensus::Peer*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_insert_bucket(std::pair, std::allocator > const, kudu::consensus::Peer*> const&, unsigned long, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:854 #4 0x20c29cf in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::Peer*>, std::allocator, std::allocator > const, kudu::consensus::Peer*> >, std::_Select1st, std::allocator > const, kudu::consensus::Peer*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::_M_insert(std::pair, std::allocator > const, kudu::consensus::Peer*> const&, std::tr1::integral_constant) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:897 #5 0x20c1fd3 in std::tr1::_Hashtable, std::allocator >, std::pair, std::allocator > const, kudu::consensus::Peer*>, std::allocator, std::allocator > const, kudu::consensus::Peer*> >, std::_Select1st, std::allocator > const, kudu::consensus::Peer*> >, std::equal_to, std::allocator > >, std::tr1::hash, std::allocator > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, false, true>::insert(std::pair, std::allocator > const, kudu::consensus::Peer*> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/tr1/hashtable.h:375 #6 0x20c1a8f in bool InsertIfNotPresent, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > > >(std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >*, std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >::value_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:344 #7 0x20c16f7 in bool InsertIfNotPresent, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > > >(std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >*, std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >::value_type::first_type const&, std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:353 #8 0x20b5675 in void InsertOrDie, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > > >(std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >*, std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >::value_type::first_type const&, std::tr1::unordered_map, std::allocator >, kudu::consensus::Peer*, std::tr1::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, kudu::consensus::Peer*> > >::value_type::second_type const&) /srv/home/alex/dev/kudu/src/kudu/gutil/map-util.h:377 #9 0x209d87a in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:435 #10 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #11 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #12 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #13 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #14 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #15 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #16 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #17 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #18 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #19 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #20 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #21 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #22 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #23 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #24 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #25 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #26 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #27 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #28 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #29 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) Indirect leak of 24 byte(s) in 3 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20b0d80 in kudu::consensus::ConsensusRequestPB::set_tablet_id(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:4156 #2 0x2093625 in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:290 #3 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #4 0x20a0d87 in kudu::consensus::PeerManager::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:467 (discriminator 1) #5 0x220b6bb in kudu::consensus::RaftConsensus::Replicate(kudu::consensus::ConsensusRound*) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:303 (discriminator 1) #6 0x191990d in kudu::tablet::TransactionDriver::PrepareAndStart() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:159 #7 0x1917012 in kudu::tablet::TransactionDriver::PrepareAndStartTask() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:114 #8 0x192ad39 in kudu::internal::RunnableAdapter::Run(kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:134 (discriminator 4) #9 0x192a908 in kudu::internal::InvokeHelper, void (kudu::tablet::TransactionDriver*)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:871 #10 0x192a58c in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::tablet::TransactionDriver*), void (kudu::internal::UnretainedWrapper)>, void (kudu::tablet::TransactionDriver*)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1063 #11 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #12 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #13 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #14 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #15 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #16 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #17 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #18 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #19 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #20 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #21 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #22 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #23 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #24 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #25 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 24 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x18b0ed9 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:118 #2 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #3 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #4 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #5 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #6 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #7 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #8 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #9 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #10 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #11 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #12 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #13 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #14 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #15 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #16 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 24 byte(s) in 3 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x35c2df9 in kudu::metadata::QuorumPeerPB::mutable_permanent_uuid() /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.h:357 #2 0x35b28c7 in kudu::metadata::QuorumPeerPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:258 #3 0x35c3ce7 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::metadata::QuorumPeerPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #4 0x35bce10 in kudu::metadata::QuorumPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:588 #5 0x3117df7 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::metadata::QuorumPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #6 0x320d441 in kudu::consensus::ConsensusMetadataPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:1411 #7 0x3eaa5ca in google::protobuf::(anonymous namespace)::InlineMergeFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/message_lite.cc:121 #8 0x208b3c0 in kudu::consensus::ConsensusMetadata::Load(kudu::FsManager*, std::basic_string, std::allocator > const&, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_meta.cc:41 (discriminator 5) #9 0x18b0503 in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:108 (discriminator 3) #10 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #11 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #12 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #13 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #17 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #18 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #19 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #20 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #21 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #22 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #23 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #24 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 16 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x369b9c0 in kudu::rpc::RequestHeader::set_method_name(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/rpc/rpc_header.pb.h:1730 #2 0x368ab35 in OutboundCall /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:40 #3 0x36e4975 in kudu::rpc::Proxy::AsyncRequest(std::basic_string, std::allocator > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, boost::function const&) const /srv/home/alex/dev/kudu/src/kudu/rpc/proxy.cc:65 #4 0x32e2c66 in kudu::consensus::ConsensusServiceProxy::UpdateConsensusAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.proxy.cc:49 (discriminator 1) #5 0x20a378e in kudu::consensus::RpcPeerProxy::UpdateAsync(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:499 #6 0x20d94ed in kudu::consensus::RemotePeer::SendRequest(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::Callback const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:177 (discriminator 3) #7 0x209413e in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:311 #8 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #9 0x20d3ad4 in boost::_mfi::mf1::operator()(kudu::consensus::Peer*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #10 0x20d351b in kudu::Status boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, long) /usr/include/boost/bind/bind.hpp:303 #11 0x20d3154 in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #12 0x20d1fb3 in boost::detail::function::function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, kudu::Status>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:132 #13 0x3763a35 in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #14 0x3b98620 in kudu::ResettableHeartbeaterThread::RunThread() /srv/home/alex/dev/kudu/src/kudu/util/resettable_heartbeater.cc:106 #15 0x3b9c9d9 in boost::_mfi::mf0::operator()(kudu::ResettableHeartbeaterThread*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #16 0x3b9c535 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #17 0x3b9c21a in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #18 0x3b9b104 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #19 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #20 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #21 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 16 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0xcfe356 in __gnu_cxx::new_allocator, std::allocator > >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0xcfc6a0 in std::_Vector_base, std::allocator >, std::allocator, std::allocator > > >::_M_allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_vector.h:168 (discriminator 13) #3 0xcfb10a in std::vector, std::allocator >, std::allocator, std::allocator > > >::_M_insert_aux(__gnu_cxx::__normal_iterator, std::allocator >*, std::vector, std::allocator >, std::allocator, std::allocator > > > >, std::basic_string, std::allocator > const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/vector.tcc:345 #4 0xc77d8a in std::vector, std::allocator >, std::allocator, std::allocator > > >::push_back(std::basic_string, std::allocator > const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_vector.h:913 #5 0x218a908 in LeaderElection /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:150 #6 0x21fe041 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:192 #7 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #8 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #9 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #10 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #11 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #12 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #13 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #14 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #15 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #16 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 16 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x369b9c0 in kudu::rpc::RequestHeader::set_method_name(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/rpc/rpc_header.pb.h:1730 #2 0x368ab35 in OutboundCall /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:40 #3 0x36e4975 in kudu::rpc::Proxy::AsyncRequest(std::basic_string, std::allocator > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, boost::function const&) const /srv/home/alex/dev/kudu/src/kudu/rpc/proxy.cc:65 #4 0x32e39b6 in kudu::consensus::ConsensusServiceProxy::RequestConsensusVoteAsync(kudu::consensus::VoteRequestPB const&, kudu::consensus::VoteResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.proxy.cc:60 (discriminator 1) #5 0x20a3c0e in kudu::consensus::RpcPeerProxy::RequestConsensusVoteAsync(kudu::consensus::VoteRequestPB const*, kudu::consensus::VoteResponsePB*, kudu::rpc::RpcController*, boost::function const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:508 #6 0x218e198 in kudu::consensus::LeaderElection::Run() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:184 (discriminator 1) #7 0x21fe247 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:197 (discriminator 1) #8 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #9 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #10 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #11 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #12 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #13 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #17 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 16 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3aa1326 in __gnu_cxx::new_allocator::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x3a9e650 in std::_Vector_base >::_M_allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_vector.h:168 (discriminator 5) #3 0x3a9ccfa in std::vector >::_M_insert_aux(__gnu_cxx::__normal_iterator > >, kudu::MemTracker* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/vector.tcc:345 #4 0x3a985ea in std::vector >::push_back(kudu::MemTracker* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_vector.h:913 #5 0x3a82f65 in kudu::MemTracker::Init() /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:363 (discriminator 1) #6 0x3a81fd4 in MemTracker /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:78 #7 0x3a80131 in kudu::MemTracker::CreateTracker(long, std::basic_string, std::allocator > const&, kudu::MemTracker*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:47 #8 0x2102860 in PeerMessageQueue /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:116 #9 0x18b103a in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 (discriminator 1) #10 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #11 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #12 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #13 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #17 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #18 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #19 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #20 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #21 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #22 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #23 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #24 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 16 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3aa1326 in __gnu_cxx::new_allocator::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x3a9e650 in std::_Vector_base >::_M_allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_vector.h:168 (discriminator 5) #3 0x3a9ccfa in std::vector >::_M_insert_aux(__gnu_cxx::__normal_iterator > >, kudu::MemTracker* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/vector.tcc:345 #4 0x3a985ea in std::vector >::push_back(kudu::MemTracker* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_vector.h:913 #5 0x3a82f04 in kudu::MemTracker::Init() /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:362 #6 0x3a81fd4 in MemTracker /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:78 #7 0x3a80131 in kudu::MemTracker::CreateTracker(long, std::basic_string, std::allocator > const&, kudu::MemTracker*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:47 #8 0x2102860 in PeerMessageQueue /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:116 #9 0x18b103a in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 (discriminator 1) #10 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #11 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #12 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #13 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #17 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #18 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #19 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #20 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #21 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #22 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #23 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #24 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 16 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x1141480 in kudu::metadata::QuorumPeerPB::set_permanent_uuid(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.h:336 #2 0x35b05d5 in kudu::metadata::QuorumPeerPB::MergeFrom(kudu::metadata::QuorumPeerPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:425 #3 0x35afe84 in QuorumPeerPB /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:187 #4 0x2091895 in Peer /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:246 (discriminator 1) #5 0x2090674 in kudu::consensus::Peer::NewRemotePeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, gscoped_ptr >, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:221 (discriminator 1) #6 0x209ea8e in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:443 (discriminator 4) #7 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #8 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #9 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #10 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #11 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #12 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #13 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #14 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #15 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #16 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #17 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #18 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #19 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #20 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #21 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #22 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #23 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #24 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #25 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #26 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #27 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #28 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #29 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 16 byte(s) in 2 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x12d5130 in kudu::HostPortPB::set_host(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/common/common.pb.h:917 #2 0x3667ea5 in kudu::HostPortPB::MergeFrom(kudu::HostPortPB const&) /srv/home/alex/dev/kudu/src/kudu/common/common.pb.cc:1279 #3 0x35b07ed in kudu::metadata::QuorumPeerPB::MergeFrom(kudu::metadata::QuorumPeerPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:431 #4 0x35afe84 in QuorumPeerPB /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:187 #5 0x2091895 in Peer /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:246 (discriminator 1) #6 0x2090674 in kudu::consensus::Peer::NewRemotePeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, gscoped_ptr >, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:221 (discriminator 1) #7 0x209ea8e in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:443 (discriminator 4) #8 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #9 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #10 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #11 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #12 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #13 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #14 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #15 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #16 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #17 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #18 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #19 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #20 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #21 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #22 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #23 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #24 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #25 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #26 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #27 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #28 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #29 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) Indirect leak of 8 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x1141480 in kudu::metadata::QuorumPeerPB::set_permanent_uuid(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.h:336 #2 0x35b05d5 in kudu::metadata::QuorumPeerPB::MergeFrom(kudu::metadata::QuorumPeerPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:425 #3 0x35afe84 in QuorumPeerPB /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:187 #4 0x2091895 in Peer /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:246 (discriminator 1) #5 0x208de9f in kudu::consensus::Peer::NewLocalPeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, kudu::log::Log*, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:203 (discriminator 1) #6 0x209d050 in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:429 (discriminator 1) #7 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #8 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #9 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #10 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #11 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #12 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #13 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #14 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #15 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #16 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #17 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #18 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #19 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #20 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #21 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #22 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #23 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #24 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #25 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #26 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #27 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #28 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #29 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 8 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x227ea20 in kudu::consensus::ConsensusMetadataPB::set_voted_for(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:2690 #2 0x226784a in kudu::consensus::ReplicaState::SetVotedForCurrentTermUnlocked(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus_state.cc:334 #3 0x21fc92e in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:179 (discriminator 4) #4 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #5 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #6 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #7 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #8 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #9 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #10 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #11 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #12 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #13 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 8 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x223b3c0 in kudu::consensus::VoteRequestPB::set_tablet_id(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:3758 #2 0x325db15 in kudu::consensus::VoteRequestPB::MergeFrom(kudu::consensus::VoteRequestPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:4420 #3 0x325d3c4 in VoteRequestPB /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:4136 #4 0x2189514 in LeaderElection /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:147 (discriminator 2) #5 0x21fe041 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:192 #6 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #7 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #8 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #9 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #10 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #11 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #12 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #13 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #14 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #15 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 8 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3aa1326 in __gnu_cxx::new_allocator::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x3a9e650 in std::_Vector_base >::_M_allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_vector.h:168 (discriminator 5) #3 0x3a9ccfa in std::vector >::_M_insert_aux(__gnu_cxx::__normal_iterator > >, kudu::MemTracker* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/vector.tcc:345 #4 0x3a985ea in std::vector >::push_back(kudu::MemTracker* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_vector.h:913 #5 0x3a82f65 in kudu::MemTracker::Init() /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:363 (discriminator 1) #6 0x3a81fd4 in MemTracker /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:78 #7 0x3a87854 in kudu::MemTracker::FindOrCreateTracker(long, std::basic_string, std::allocator > const&, kudu::MemTracker*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:136 #8 0x21024b7 in PeerMessageQueue /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:112 #9 0x18b103a in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 (discriminator 1) #10 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #11 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #12 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #13 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #17 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #18 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #19 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #20 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #21 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #22 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #23 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #24 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 8 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x223ad00 in kudu::consensus::VoteRequestPB::set_candidate_uuid(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:3828 #2 0x325dcab in kudu::consensus::VoteRequestPB::MergeFrom(kudu::consensus::VoteRequestPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:4423 #3 0x325d3c4 in VoteRequestPB /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:4136 #4 0x2189514 in LeaderElection /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:147 (discriminator 2) #5 0x21fe041 in kudu::consensus::RaftConsensus::StartElection() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:192 #6 0x1428576 in kudu::tserver::ConsensusServiceImpl::MakePeerLeader(kudu::consensus::MakePeerLeaderRequestPB const*, kudu::consensus::MakePeerLeaderResponsePB*, kudu::rpc::RpcContext*) /srv/home/alex/dev/kudu/src/kudu/tserver/tablet_service.cc:493 (discriminator 2) #7 0x32dfa4d in kudu::consensus::ConsensusServiceIf::Handle(kudu::rpc::InboundCall*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.service.cc:115 (discriminator 1) #8 0x3784a13 in kudu::rpc::ServicePool::RunThread() /srv/home/alex/dev/kudu/src/kudu/rpc/service_pool.cc:156 (discriminator 2) #9 0x379b799 in boost::_mfi::mf0::operator()(kudu::rpc::ServicePool*) const /usr/include/boost/bind/mem_fn_template.hpp:49 (discriminator 4) #10 0x379b2f5 in void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #11 0x379afda in boost::_bi::bind_t, boost::_bi::list1 > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #12 0x3799ec4 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #13 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #14 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #15 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 8 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x35f8319 in kudu::AppStatusPB::mutable_message() /srv/home/alex/dev/kudu/src/kudu/common/wire_protocol.pb.h:811 #2 0x35cb069 in kudu::AppStatusPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/common/wire_protocol.pb.cc:397 #3 0x1f36b57 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::AppStatusPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #4 0x31fed5b in kudu::consensus::ConsensusErrorPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:910 #5 0x32ce4c7 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::consensus::ConsensusErrorPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #6 0x3258bb6 in kudu::consensus::ConsensusStatusPB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:3949 #7 0x32d0137 in bool google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(google::protobuf::io::CodedInputStream*, kudu::consensus::ConsensusStatusPB*) /srv/home/alex/dev/kudu/thirdparty/installed/include/google/protobuf/wire_format_lite_inl.h:402 #8 0x3287a60 in kudu::consensus::ConsensusResponsePB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:5448 #9 0x3eaa5ca in google::protobuf::(anonymous namespace)::InlineMergeFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/message_lite.cc:121 #10 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #11 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #12 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #13 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 8 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x32d3d69 in kudu::consensus::ConsensusResponsePB::mutable_responder_uuid() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:4444 #2 0x32866e6 in kudu::consensus::ConsensusResponsePB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:5418 #3 0x3eaa5ca in google::protobuf::(anonymous namespace)::InlineMergeFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/message_lite.cc:121 #4 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #5 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #6 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #7 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 Indirect leak of 8 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x20fdba0 in kudu::consensus::ConsensusResponsePB::set_responder_uuid(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:4423 #2 0x20e643f in kudu::consensus::LocalPeer::SetupResponse(kudu::consensus::ConsensusResponsePB*, unsigned long) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:122 #3 0x20e354e in kudu::consensus::LocalPeer::SendRequest(kudu::consensus::ConsensusRequestPB const*, kudu::consensus::ConsensusResponsePB*, kudu::Callback const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:89 #4 0x209413e in kudu::consensus::Peer::SendNextRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:311 #5 0x2092eb1 in kudu::consensus::Peer::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:283 #6 0x20a0d87 in kudu::consensus::PeerManager::SignalRequest(bool) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:467 (discriminator 1) #7 0x220b6bb in kudu::consensus::RaftConsensus::Replicate(kudu::consensus::ConsensusRound*) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:303 (discriminator 1) #8 0x191990d in kudu::tablet::TransactionDriver::PrepareAndStart() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:159 #9 0x1917012 in kudu::tablet::TransactionDriver::PrepareAndStartTask() /srv/home/alex/dev/kudu/src/kudu/tablet/transactions/transaction_driver.cc:114 #10 0x192ad39 in kudu::internal::RunnableAdapter::Run(kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:134 (discriminator 4) #11 0x192a908 in kudu::internal::InvokeHelper, void (kudu::tablet::TransactionDriver*)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::tablet::TransactionDriver*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:871 #12 0x192a58c in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::tablet::TransactionDriver*), void (kudu::internal::UnretainedWrapper)>, void (kudu::tablet::TransactionDriver*)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1063 #13 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #14 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #15 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #16 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #17 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #18 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #19 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #20 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #21 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #22 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #23 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #24 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #25 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #26 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #27 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 8 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x3aa1326 in __gnu_cxx::new_allocator::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:104 #2 0x3a9e650 in std::_Vector_base >::_M_allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_vector.h:168 (discriminator 5) #3 0x3a9ccfa in std::vector >::_M_insert_aux(__gnu_cxx::__normal_iterator > >, kudu::MemTracker* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/vector.tcc:345 #4 0x3a985ea in std::vector >::push_back(kudu::MemTracker* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_vector.h:913 #5 0x3a82f04 in kudu::MemTracker::Init() /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:362 #6 0x3a81fd4 in MemTracker /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:78 #7 0x3a87854 in kudu::MemTracker::FindOrCreateTracker(long, std::basic_string, std::allocator > const&, kudu::MemTracker*) /srv/home/alex/dev/kudu/src/kudu/util/mem_tracker.cc:136 #8 0x21024b7 in PeerMessageQueue /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_queue.cc:112 #9 0x18b103a in kudu::tablet::TabletPeer::Init(std::tr1::shared_ptr const&, scoped_refptr const&, std::tr1::shared_ptr const&, gscoped_ptr >, kudu::MetricContext const&) /srv/home/alex/dev/kudu/src/kudu/tablet/tablet_peer.cc:123 (discriminator 1) #10 0x1486614 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr const&) /srv/home/alex/dev/kudu/src/kudu/tserver/ts_tablet_manager.cc:352 #11 0x14d94cb in boost::_mfi::mf1 const&>::operator()(kudu::tserver::TSTabletManager*, scoped_refptr const&) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #12 0x14d8f99 in void boost::_bi::list2, boost::_bi::value > >::operator() const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #13 0x14d8c1a in boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #14 0x14d76c1 in boost::detail::function::void_function_obj_invoker0 const&>, boost::_bi::list2, boost::_bi::value > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #15 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #16 0x3c33db2 in kudu::FunctionRunnable::Run() /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:34 #17 0x3c20e33 in kudu::ThreadPool::DispatchThread(bool) /srv/home/alex/dev/kudu/src/kudu/util/threadpool.cc:272 (discriminator 1) #18 0x3c2a29b in boost::_mfi::mf1::operator()(kudu::ThreadPool*, bool) const /usr/include/boost/bind/mem_fn_template.hpp:165 (discriminator 4) #19 0x3c29d15 in void boost::_bi::list2, boost::_bi::value >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:313 #20 0x3c2994a in boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #21 0x3c287d1 in boost::detail::function::void_function_obj_invoker0, boost::_bi::list2, boost::_bi::value > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #22 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #23 0x3bd2a4b in kudu::Thread::SuperviseThread(void*) /srv/home/alex/dev/kudu/src/kudu/util/thread.cc:436 #24 0x7f7f2e48b181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2) Indirect leak of 8 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x12d5130 in kudu::HostPortPB::set_host(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/common/common.pb.h:917 #2 0x3667ea5 in kudu::HostPortPB::MergeFrom(kudu::HostPortPB const&) /srv/home/alex/dev/kudu/src/kudu/common/common.pb.cc:1279 #3 0x35b07ed in kudu::metadata::QuorumPeerPB::MergeFrom(kudu::metadata::QuorumPeerPB const&) /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:431 #4 0x35afe84 in QuorumPeerPB /srv/home/alex/dev/kudu/src/kudu/server/metadata.pb.cc:187 #5 0x2091895 in Peer /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:246 (discriminator 1) #6 0x208de9f in kudu::consensus::Peer::NewLocalPeer(kudu::metadata::QuorumPeerPB const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, kudu::consensus::PeerMessageQueue*, kudu::log::Log*, gscoped_ptr >*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:203 (discriminator 1) #7 0x209d050 in kudu::consensus::PeerManager::UpdateQuorum(kudu::metadata::QuorumPB const&) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus_peers.cc:429 (discriminator 1) #8 0x2204b3e in kudu::consensus::RaftConsensus::BecomeLeaderUnlocked() /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:237 (discriminator 1) #9 0x2203413 in kudu::consensus::RaftConsensus::ElectionCallback(kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/consensus/raft_consensus.cc:1027 (discriminator 1) #10 0x2242633 in kudu::internal::RunnableAdapter::Run(kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #11 0x2242175 in kudu::internal::InvokeHelper, void (kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&)>::MakeItSo(kudu::internal::RunnableAdapter, kudu::consensus::RaftConsensus* const&, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #12 0x2241d3b in kudu::internal::Invoker<1, kudu::internal::BindState, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&), void (kudu::consensus::RaftConsensus*)>, void (kudu::consensus::RaftConsensus*, kudu::consensus::ElectionResult const&)>::Run(kudu::internal::BindStateBase*, kudu::consensus::ElectionResult const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1116 #13 0x21a1e8e in kudu::Callback::Run(kudu::consensus::ElectionResult const&) const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:436 #14 0x2190ae0 in kudu::consensus::LeaderElection::InvokeDecisionCallbackIfNeeded() /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:207 #15 0x21904dd in kudu::consensus::LeaderElection::VoteResponseRpcCallback(std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/consensus/leader_election.cc:247 #16 0x21ac133 in kudu::internal::RunnableAdapter, std::allocator > const&)>::Run(kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:190 #17 0x21abb15 in kudu::internal::InvokeHelper, std::allocator > const&)>, void (kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&)>::MakeItSo(kudu::internal::RunnableAdapter, std::allocator > const&)>, kudu::consensus::LeaderElection* const&, std::basic_string, std::allocator > const&) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:887 #18 0x21ab575 in kudu::internal::Invoker<2, kudu::internal::BindState, std::allocator > const&)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&), void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator >)>, void (kudu::consensus::LeaderElection*, std::basic_string, std::allocator > const&)>::Run(kudu::internal::BindStateBase*) /srv/home/alex/dev/kudu/src/kudu/gutil/bind_internal.h:1145 #19 0x21a0e07 in kudu::Callback::Run() const /srv/home/alex/dev/kudu/src/kudu/gutil/callback.h:396 #20 0x21a7519 in boost::_mfi::cmf0 >::operator()(kudu::Callback const&) const /usr/include/boost/bind/mem_fn_template.hpp:120 (discriminator 4) #21 0x21a7080 in void boost::_bi::list1 > >::operator() >, boost::_bi::list0>(boost::_bi::type, boost::_mfi::cmf0 >&, boost::_bi::list0&, int) /usr/include/boost/bind/bind.hpp:253 #22 0x21a6d8a in boost::_bi::bind_t >, boost::_bi::list1 > > >::operator()() /usr/include/boost/bind/bind_template.hpp:20 #23 0x21a5831 in boost::detail::function::void_function_obj_invoker0 >, boost::_bi::list1 > > >, void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:153 #24 0x193977e in boost::function0::operator()() const /usr/include/boost/function/function_template.hpp:766 #25 0x3692892 in kudu::rpc::OutboundCall::CallCallback() /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:140 #26 0x3693022 in kudu::rpc::OutboundCall::SetResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/outbound_call.cc:157 #27 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #28 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #29 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) Indirect leak of 8 byte(s) in 1 object(s) allocated from: #0 0xafd0ab in operator new(unsigned long) ??:? #1 0x32d1409 in kudu::consensus::VoteResponsePB::mutable_responder_uuid() /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.h:3983 #2 0x326aebd in kudu::consensus::VoteResponsePB::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*) /srv/home/alex/dev/kudu/src/kudu/consensus/consensus.pb.cc:4578 #3 0x3eaa5ca in google::protobuf::(anonymous namespace)::InlineMergeFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) /srv/home/alex/dev/kudu/thirdparty/protobuf-2.5.0/src/google/protobuf/message_lite.cc:121 #4 0x37ba53c in kudu::rpc::Connection::HandleCallResponse(gscoped_ptr >) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:459 (discriminator 5) #5 0x37b780a in kudu::rpc::Connection::ReadHandler(ev::io&, int) /srv/home/alex/dev/kudu/src/kudu/rpc/connection.cc:395 (discriminator 3) #6 0x37f9e2e in void ev::base::method_thunk(ev_loop*, ev_io*, int) /srv/home/alex/dev/kudu/thirdparty/installed/include/ev++.h:479 (discriminator 8) #7 0x3924834 in ev_invoke_pending /srv/home/alex/dev/kudu/thirdparty/libev-4.15/ev.c:2994 SUMMARY: AddressSanitizer: 12840 byte(s) leaked in 197 allocation(s).