Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-1362

Flaky test: SlaveRecoveryTest/0.RemoveNonCheckpointingFramework

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.19.0
    • 0.19.0
    • test
    • None
    • Q2'14 Sprint 2

    Description

      [ RUN      ] SlaveRecoveryTest/0.RemoveNonCheckpointingFramework
      Using temporary directory '/tmp/SlaveRecoveryTest_0_RemoveNonCheckpointingFramework_lneUgg'
      I0513 22:32:49.143599 13118 leveldb.cpp:176] Opened db in 145.113217ms
      I0513 22:32:49.165134 13118 leveldb.cpp:183] Compacted db in 21.474397ms
      I0513 22:32:49.165197 13118 leveldb.cpp:198] Created db iterator in 6280ns
      I0513 22:32:49.165225 13118 leveldb.cpp:204] Seeked to beginning of db in 1070ns
      I0513 22:32:49.165246 13118 leveldb.cpp:273] Iterated through 0 keys in the db in 550ns
      I0513 22:32:49.165305 13118 replica.cpp:741] Replica recovered with log positions 0 -> 0 with 1 holes and 0 unlearned
      I0513 22:32:49.166082 13149 recover.cpp:425] Starting replica recovery
      I0513 22:32:49.166543 13144 recover.cpp:451] Replica is in EMPTY status
      I0513 22:32:49.168061 13154 replica.cpp:638] Replica in EMPTY status received a broadcasted recover request
      I0513 22:32:49.168519 13143 recover.cpp:188] Received a recover response from a replica in EMPTY status
      I0513 22:32:49.168859 13141 master.cpp:267] Master 20140513-223249-1740121354-38066-13118 (smfd-bkq-03-sr4.devel.twitter.com) started on 10.37.184.103:38066
      I0513 22:32:49.168905 13141 master.cpp:304] Master only allowing authenticated frameworks to register
      I0513 22:32:49.168920 13141 master.cpp:309] Master only allowing authenticated slaves to register
      I0513 22:32:49.168936 13141 credentials.hpp:35] Loading credentials for authentication
      I0513 22:32:49.169141 13145 recover.cpp:542] Updating replica status to STARTING
      W0513 22:32:49.169212 13141 credentials.hpp:48] Failed to stat credentials file 'file:///tmp/SlaveRecoveryTest_0_RemoveNonCheckpointingFramework_lneUgg/credentials': No such file or directory
      I0513 22:32:49.171468 13149 master.cpp:919] The newly elected leader is master@10.37.184.103:38066 with id 20140513-223249-1740121354-38066-13118
      I0513 22:32:49.171488 13149 master.cpp:929] Elected as the leading master!
      I0513 22:32:49.171500 13149 master.cpp:750] Recovering from registrar
      I0513 22:32:49.171622 13150 registrar.cpp:313] Recovering registrar
      I0513 22:32:49.210352 13144 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 40.492309ms
      I0513 22:32:49.210404 13144 replica.cpp:320] Persisted replica status to STARTING
      I0513 22:32:49.210630 13144 recover.cpp:451] Replica is in STARTING status
      I0513 22:32:49.211668 13133 replica.cpp:638] Replica in STARTING status received a broadcasted recover request
      I0513 22:32:49.212237 13155 recover.cpp:188] Received a recover response from a replica in STARTING status
      I0513 22:32:49.212793 13151 recover.cpp:542] Updating replica status to VOTING
      I0513 22:32:49.231677 13145 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 18.50995ms
      I0513 22:32:49.231758 13145 replica.cpp:320] Persisted replica status to VOTING
      I0513 22:32:49.232009 13133 recover.cpp:556] Successfully joined the Paxos group
      I0513 22:32:49.232292 13133 recover.cpp:440] Recover process terminated
      I0513 22:32:49.233135 13138 log.cpp:656] Attempting to start the writer
      I0513 22:32:49.235530 13146 replica.cpp:474] Replica received implicit promise request with proposal 1
      I0513 22:32:49.240046 13146 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 4.477846ms
      I0513 22:32:49.240084 13146 replica.cpp:342] Persisted promised to 1
      I0513 22:32:49.241796 13150 coordinator.cpp:230] Coordinator attemping to fill missing position
      I0513 22:32:49.243787 13134 replica.cpp:375] Replica received explicit promise request for position 0 with proposal 2
      I0513 22:32:49.248342 13134 leveldb.cpp:343] Persisting action (8 bytes) to leveldb took 4.509976ms
      I0513 22:32:49.248392 13134 replica.cpp:676] Persisted action at 0
      I0513 22:32:49.249868 13133 replica.cpp:508] Replica received write request for position 0
      I0513 22:32:49.249922 13133 leveldb.cpp:438] Reading position from leveldb took 18826ns
      I0513 22:32:49.256672 13133 leveldb.cpp:343] Persisting action (14 bytes) to leveldb took 6.713303ms
      I0513 22:32:49.256716 13133 replica.cpp:676] Persisted action at 0
      I0513 22:32:49.257447 13134 replica.cpp:655] Replica received learned notice for position 0
      I0513 22:32:49.265025 13134 leveldb.cpp:343] Persisting action (16 bytes) to leveldb took 7.5032ms
      I0513 22:32:49.265074 13134 replica.cpp:676] Persisted action at 0
      I0513 22:32:49.265099 13134 replica.cpp:661] Replica learned NOP action at position 0
      I0513 22:32:49.265629 13136 log.cpp:672] Writer started with ending position 0
      I0513 22:32:49.267151 13153 leveldb.cpp:438] Reading position from leveldb took 41406ns
      I0513 22:32:49.269865 13147 registrar.cpp:346] Successfully fetched the registry (0B)
      I0513 22:32:49.269969 13147 registrar.cpp:422] Attempting to update the 'registry'
      I0513 22:32:49.272371 13142 log.cpp:680] Attempting to append 155 bytes to the log
      I0513 22:32:49.272488 13152 coordinator.cpp:340] Coordinator attempting to write APPEND action at position 1
      I0513 22:32:49.273243 13153 replica.cpp:508] Replica received write request for position 1
      I0513 22:32:49.281638 13153 leveldb.cpp:343] Persisting action (174 bytes) to leveldb took 8.365105ms
      I0513 22:32:49.281680 13153 replica.cpp:676] Persisted action at 1
      I0513 22:32:49.282948 13149 replica.cpp:655] Replica received learned notice for position 1
      I0513 22:32:49.290007 13149 leveldb.cpp:343] Persisting action (176 bytes) to leveldb took 6.900024ms
      I0513 22:32:49.290096 13149 replica.cpp:676] Persisted action at 1
      I0513 22:32:49.290123 13149 replica.cpp:661] Replica learned APPEND action at position 1
      I0513 22:32:49.291050 13145 registrar.cpp:479] Successfully updated 'registry'
      I0513 22:32:49.291299 13145 registrar.cpp:372] Successfully recovered registrar
      I0513 22:32:49.291355 13147 log.cpp:699] Attempting to truncate the log to 1
      I0513 22:32:49.291687 13141 master.cpp:777] Recovered 0 slaves from the Registry (117B) ; allowing 10mins for slaves to re-register
      I0513 22:32:49.291723 13153 coordinator.cpp:340] Coordinator attempting to write TRUNCATE action at position 2
      I0513 22:32:49.293115 13136 replica.cpp:508] Replica received write request for position 2
      I0513 22:32:49.294591 13118 mesos_containerizer.cpp:122] Using isolation: cgroups/cpu,cgroups/mem
      I0513 22:32:49.298322 13136 leveldb.cpp:343] Persisting action (16 bytes) to leveldb took 5.160841ms
      I0513 22:32:49.298372 13136 replica.cpp:676] Persisted action at 2
      I0513 22:32:49.299250 13145 replica.cpp:655] Replica received learned notice for position 2
      I0513 22:32:49.306643 13145 leveldb.cpp:343] Persisting action (18 bytes) to leveldb took 7.337541ms
      I0513 22:32:49.306712 13145 leveldb.cpp:401] Deleting ~1 keys from leveldb took 22147ns
      I0513 22:32:49.306748 13145 replica.cpp:676] Persisted action at 2
      I0513 22:32:49.306778 13145 replica.cpp:661] Replica learned TRUNCATE action at position 2
      I0513 22:32:49.436813 13118 linux_launcher.cpp:66] Using /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
      I0513 22:32:49.440974 13141 slave.cpp:143] Slave started on 14)@10.37.184.103:38066
      I0513 22:32:49.441046 13141 slave.cpp:152] Moving slave process into its own cgroup
      I0513 22:32:49.444485 13118 sched.cpp:121] Version: 0.19.0
      I0513 22:32:49.445209 13142 sched.cpp:217] New master detected at master@10.37.184.103:38066
      I0513 22:32:49.445278 13142 sched.cpp:268] Authenticating with master master@10.37.184.103:38066
      I0513 22:32:49.445497 13146 authenticatee.hpp:128] Creating new client SASL connection
      I0513 22:32:49.445678 13143 master.cpp:2803] Authenticating scheduler(8)@10.37.184.103:38066
      I0513 22:32:49.445858 13148 authenticator.hpp:148] Creating new server SASL connection
      I0513 22:32:49.445971 13143 authenticatee.hpp:219] Received SASL authentication mechanisms: CRAM-MD5
      I0513 22:32:49.446002 13143 authenticatee.hpp:245] Attempting to authenticate with mechanism 'CRAM-MD5'
      I0513 22:32:49.446117 13149 authenticator.hpp:254] Received SASL authentication start
      I0513 22:32:49.446182 13149 authenticator.hpp:342] Authentication requires more steps
      I0513 22:32:49.446254 13149 authenticatee.hpp:265] Received SASL authentication step
      I0513 22:32:49.446351 13149 authenticator.hpp:282] Received SASL authentication step
      I0513 22:32:49.446437 13149 authenticator.hpp:334] Authentication success
      I0513 22:32:49.446583 13149 authenticatee.hpp:305] Authentication success
      I0513 22:32:49.446616 13150 master.cpp:2843] Successfully authenticated scheduler(8)@10.37.184.103:38066
      I0513 22:32:49.446851 13152 sched.cpp:342] Successfully authenticated with master master@10.37.184.103:38066
      I0513 22:32:49.447042 13150 master.cpp:978] Received registration request from scheduler(8)@10.37.184.103:38066
      I0513 22:32:49.447113 13150 master.cpp:996] Registering framework 20140513-223249-1740121354-38066-13118-0000 at scheduler(8)@10.37.184.103:38066
      I0513 22:32:49.447293 13150 sched.cpp:392] Framework registered with 20140513-223249-1740121354-38066-13118-0000
      I0513 22:32:49.447383 13147 hierarchical_allocator_process.hpp:331] Added framework 20140513-223249-1740121354-38066-13118-0000
      I0513 22:32:49.449103 13141 slave.cpp:152] Moving slave process into its own cgroup
      I0513 22:32:49.454275 13141 credentials.hpp:35] Loading credentials for authentication
      W0513 22:32:49.454345 13141 credentials.hpp:48] Failed to stat credentials file 'file:///tmp/SlaveRecoveryTest_0_RemoveNonCheckpointingFramework_zqhtfa/credential': No such file or directory
      I0513 22:32:49.454396 13141 slave.cpp:233] Slave using credential for: test-principal
      I0513 22:32:49.454560 13141 slave.cpp:246] Slave resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
      I0513 22:32:49.455066 13141 slave.cpp:274] Slave hostname: smfd-bkq-03-sr4.devel.twitter.com
      I0513 22:32:49.455117 13141 slave.cpp:275] Slave checkpoint: true
      I0513 22:32:49.455879 13136 state.cpp:33] Recovering state from '/tmp/SlaveRecoveryTest_0_RemoveNonCheckpointingFramework_zqhtfa/meta'
      I0513 22:32:49.456261 13152 status_update_manager.cpp:193] Recovering status update manager
      I0513 22:32:49.456564 13148 mesos_containerizer.cpp:279] Recovering containerizer
      I0513 22:32:49.458384 13144 mem.cpp:165] Removing orphaned cgroup 'mesos_test_f73e20ca-48d7-42a9-8f84-ecb1c85163ab/slave'
      I0513 22:32:49.458623 13151 cpushare.cpp:189] Removing orphaned cgroup 'cpuacct/mesos_test_f73e20ca-48d7-42a9-8f84-ecb1c85163ab/slave'
      I0513 22:32:49.461091 13135 slave.cpp:2962] Finished recovery
      I0513 22:32:49.461506 13135 slave.cpp:527] New master detected at master@10.37.184.103:38066
      I0513 22:32:49.461647 13135 slave.cpp:587] Authenticating with master master@10.37.184.103:38066
      I0513 22:32:49.461678 13152 status_update_manager.cpp:167] New master detected at master@10.37.184.103:38066
      I0513 22:32:49.462107 13135 slave.cpp:560] Detecting new master
      I0513 22:32:49.462165 13146 authenticatee.hpp:128] Creating new client SASL connection
      I0513 22:32:49.462525 13136 master.cpp:2803] Authenticating slave(14)@10.37.184.103:38066
      I0513 22:32:49.462728 13133 authenticator.hpp:148] Creating new server SASL connection
      I0513 22:32:49.462879 13133 authenticatee.hpp:219] Received SASL authentication mechanisms: CRAM-MD5
      I0513 22:32:49.462910 13133 authenticatee.hpp:245] Attempting to authenticate with mechanism 'CRAM-MD5'
      I0513 22:32:49.463008 13146 authenticator.hpp:254] Received SASL authentication start
      I0513 22:32:49.463078 13146 authenticator.hpp:342] Authentication requires more steps
      I0513 22:32:49.463181 13140 authenticatee.hpp:265] Received SASL authentication step
      I0513 22:32:49.463279 13138 authenticator.hpp:282] Received SASL authentication step
      I0513 22:32:49.463346 13138 authenticator.hpp:334] Authentication success
      I0513 22:32:49.463412 13153 authenticatee.hpp:305] Authentication success
      I0513 22:32:49.463553 13153 master.cpp:2843] Successfully authenticated slave(14)@10.37.184.103:38066
      I0513 22:32:49.463804 13144 slave.cpp:644] Successfully authenticated with master master@10.37.184.103:38066
      I0513 22:32:49.464196 13153 master.cpp:2134] Registering slave at slave(14)@10.37.184.103:38066 (smfd-bkq-03-sr4.devel.twitter.com) with id 20140513-223249-1740121354-38066-13118-0
      I0513 22:32:49.464462 13141 registrar.cpp:422] Attempting to update the 'registry'
      I0513 22:32:49.466845 13155 log.cpp:680] Attempting to append 382 bytes to the log
      I0513 22:32:49.466958 13148 coordinator.cpp:340] Coordinator attempting to write APPEND action at position 3
      I0513 22:32:49.467793 13141 replica.cpp:508] Replica received write request for position 3
      I0513 22:32:49.489850 13141 leveldb.cpp:343] Persisting action (401 bytes) to leveldb took 22.010594ms
      I0513 22:32:49.489894 13141 replica.cpp:676] Persisted action at 3
      I0513 22:32:49.490785 13151 replica.cpp:655] Replica received learned notice for position 3
      I0513 22:32:49.498225 13151 leveldb.cpp:343] Persisting action (403 bytes) to leveldb took 7.399267ms
      I0513 22:32:49.498316 13151 replica.cpp:676] Persisted action at 3
      I0513 22:32:49.498353 13151 replica.cpp:661] Replica learned APPEND action at position 3
      I0513 22:32:49.499547 13134 registrar.cpp:479] Successfully updated 'registry'
      I0513 22:32:49.499881 13144 master.cpp:2174] Registered slave 20140513-223249-1740121354-38066-13118-0 at slave(14)@10.37.184.103:38066 (smfd-bkq-03-sr4.devel.twitter.com)
      I0513 22:32:49.499938 13144 master.cpp:3288] Adding slave 20140513-223249-1740121354-38066-13118-0 at slave(14)@10.37.184.103:38066 (smfd-bkq-03-sr4.devel.twitter.com) with cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
      I0513 22:32:49.500069 13153 log.cpp:699] Attempting to truncate the log to 3
      I0513 22:32:49.500610 13147 slave.cpp:677] Registered with master master@10.37.184.103:38066; given slave ID 20140513-223249-1740121354-38066-13118-0
      I0513 22:32:49.500674 13152 coordinator.cpp:340] Coordinator attempting to write TRUNCATE action at position 4
      I0513 22:32:49.500855 13147 slave.cpp:690] Checkpointing SlaveInfo to '/tmp/SlaveRecoveryTest_0_RemoveNonCheckpointingFramework_zqhtfa/meta/slaves/20140513-223249-1740121354-38066-13118-0/slave.info'
      I0513 22:32:49.500957 13149 hierarchical_allocator_process.hpp:444] Added slave 20140513-223249-1740121354-38066-13118-0 (smfd-bkq-03-sr4.devel.twitter.com) with cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] (and cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] available)
      I0513 22:32:49.501490 13138 master.cpp:2752] Sending 1 offers to framework 20140513-223249-1740121354-38066-13118-0000
      I0513 22:32:49.502202 13150 replica.cpp:508] Replica received write request for position 4
      W0513 22:32:49.506292 13140 sched.cpp:902] Attempting to launch task 39652db7-efbf-4f2f-af8e-011e56546c67 with an unknown offer 20140513-223249-1740121354-38066-13118-0
      I0513 22:32:49.506574 13150 leveldb.cpp:343] Persisting action (16 bytes) to leveldb took 4.334236ms
      I0513 22:32:49.506656 13150 replica.cpp:676] Persisted action at 4
      I0513 22:32:49.506772 13140 master.cpp:1810] Processing reply for offers: [ 20140513-223249-1740121354-38066-13118-0 ] on slave 20140513-223249-1740121354-38066-13118-0 at slave(14)@10.37.184.103:38066 (smfd-bkq-03-sr4.devel.twitter.com) for framework 20140513-223249-1740121354-38066-13118-0000
      I0513 22:32:49.506954 13140 master.hpp:608] Adding task 246cd72c-92f1-45d4-8102-26705a8d4cd1 with resources cpus(*):1; mem(*):512 on slave 20140513-223249-1740121354-38066-13118-0 (smfd-bkq-03-sr4.devel.twitter.com)
      I0513 22:32:49.507019 13140 master.cpp:2927] Launching task 246cd72c-92f1-45d4-8102-26705a8d4cd1 of framework 20140513-223249-1740121354-38066-13118-0000 with resources cpus(*):1; mem(*):512 on slave 20140513-223249-1740121354-38066-13118-0 at slave(14)@10.37.184.103:38066 (smfd-bkq-03-sr4.devel.twitter.com)
      I0513 22:32:49.507247 13142 replica.cpp:655] Replica received learned notice for position 4
      I0513 22:32:49.507272 13140 master.hpp:608] Adding task 39652db7-efbf-4f2f-af8e-011e56546c67 with resources cpus(*):1; mem(*):512 on slave 20140513-223249-1740121354-38066-13118-0 (smfd-bkq-03-sr4.devel.twitter.com)
      I0513 22:32:49.507316 13138 slave.cpp:907] Got assigned task 246cd72c-92f1-45d4-8102-26705a8d4cd1 for framework 20140513-223249-1740121354-38066-13118-0000
      I0513 22:32:49.507340 13140 master.cpp:2927] Launching task 39652db7-efbf-4f2f-af8e-011e56546c67 of framework 20140513-223249-1740121354-38066-13118-0000 with resources cpus(*):1; mem(*):512 on slave 20140513-223249-1740121354-38066-13118-0 at slave(14)@10.37.184.103:38066 (smfd-bkq-03-sr4.devel.twitter.com)
      I0513 22:32:49.507707 13138 slave.cpp:907] Got assigned task 39652db7-efbf-4f2f-af8e-011e56546c67 for framework 20140513-223249-1740121354-38066-13118-0000
      I0513 22:32:49.507753 13150 hierarchical_allocator_process.hpp:589] Framework 20140513-223249-1740121354-38066-13118-0000 filtered slave 20140513-223249-1740121354-38066-13118-0 for 5secs
      I0513 22:32:49.507948 13138 slave.cpp:1017] Launching task 246cd72c-92f1-45d4-8102-26705a8d4cd1 for framework 20140513-223249-1740121354-38066-13118-0000
      I0513 22:32:49.510450 13154 mesos_containerizer.cpp:523] Starting container 'b3fa6e15-9b2e-4d1e-a850-b4ce83ca497b' for executor '246cd72c-92f1-45d4-8102-26705a8d4cd1' of framework '20140513-223249-1740121354-38066-13118-0000'
      I0513 22:32:49.510470 13138 slave.cpp:1127] Queuing task '246cd72c-92f1-45d4-8102-26705a8d4cd1' for executor 246cd72c-92f1-45d4-8102-26705a8d4cd1 of framework '20140513-223249-1740121354-38066-13118-0000
      I0513 22:32:49.510555 13138 slave.cpp:1017] Launching task 39652db7-efbf-4f2f-af8e-011e56546c67 for framework 20140513-223249-1740121354-38066-13118-0000
      I0513 22:32:49.512554 13151 mem.cpp:413] Started listening for OOM events for container b3fa6e15-9b2e-4d1e-a850-b4ce83ca497b
      I0513 22:32:49.512958 13140 mesos_containerizer.cpp:523] Starting container '69fd0ab4-ab01-4ec3-951f-0eab8a9fbe85' for executor '39652db7-efbf-4f2f-af8e-011e56546c67' of framework '20140513-223249-1740121354-38066-13118-0000'
      I0513 22:32:49.512980 13138 slave.cpp:1127] Queuing task '39652db7-efbf-4f2f-af8e-011e56546c67' for executor 39652db7-efbf-4f2f-af8e-011e56546c67 of framework '20140513-223249-1740121354-38066-13118-0000
      I0513 22:32:49.513157 13151 mem.cpp:277] Updated 'memory.soft_limit_in_bytes' to 512MB for container b3fa6e15-9b2e-4d1e-a850-b4ce83ca497b
      I0513 22:32:49.513746 13133 cpushare.cpp:334] Updated 'cpu.shares' to 1024 (cpus 1) for container b3fa6e15-9b2e-4d1e-a850-b4ce83ca497b
      I0513 22:32:49.514400 13151 mem.cpp:307] Updated 'memory.limit_in_bytes' to 512MB for container b3fa6e15-9b2e-4d1e-a850-b4ce83ca497b
      I0513 22:32:49.514842 13142 leveldb.cpp:343] Persisting action (18 bytes) to leveldb took 7.556298ms
      I0513 22:32:49.514960 13142 leveldb.cpp:401] Deleting ~2 keys from leveldb took 28304ns
      I0513 22:32:49.514984 13142 replica.cpp:676] Persisted action at 4
      I0513 22:32:49.515020 13142 replica.cpp:661] Replica learned TRUNCATE action at position 4
      I0513 22:32:49.516376 13148 linux_launcher.cpp:222] Cloning child process with flags = 0
      I0513 22:32:49.517678 13151 mem.cpp:413] Started listening for OOM events for container 69fd0ab4-ab01-4ec3-951f-0eab8a9fbe85
      I0513 22:32:49.525169 13151 mem.cpp:277] Updated 'memory.soft_limit_in_bytes' to 512MB for container 69fd0ab4-ab01-4ec3-951f-0eab8a9fbe85
      I0513 22:32:49.525185 13133 cpushare.cpp:334] Updated 'cpu.shares' to 1024 (cpus 1) for container 69fd0ab4-ab01-4ec3-951f-0eab8a9fbe85
      I0513 22:32:49.526921 13151 mem.cpp:307] Updated 'memory.limit_in_bytes' to 512MB for container 69fd0ab4-ab01-4ec3-951f-0eab8a9fbe85
      I0513 22:32:49.529346 13139 linux_launcher.cpp:222] Cloning child process with flags = 0
      I0513 22:32:49.584899 13139 mesos_containerizer.cpp:623] Fetching URIs for container 'b3fa6e15-9b2e-4d1e-a850-b4ce83ca497b' using command '/home/bmahler/git/mesos/build/src/mesos-fetcher'
      I0513 22:32:49.596926 13139 mesos_containerizer.cpp:623] Fetching URIs for container '69fd0ab4-ab01-4ec3-951f-0eab8a9fbe85' using command '/home/bmahler/git/mesos/build/src/mesos-fetcher'
      I0513 22:32:50.592658 13132 slave.cpp:2299] Monitoring executor '246cd72c-92f1-45d4-8102-26705a8d4cd1' of framework '20140513-223249-1740121354-38066-13118-0000' in container 'b3fa6e15-9b2e-4d1e-a850-b4ce83ca497b'
      I0513 22:32:50.593384 13132 slave.cpp:2299] Monitoring executor '39652db7-efbf-4f2f-af8e-011e56546c67' of framework '20140513-223249-1740121354-38066-13118-0000' in container '69fd0ab4-ab01-4ec3-951f-0eab8a9fbe85'
      WARNING: Logging before InitGoogleLogging() is written to STDERR
      I0513 22:32:50.643218 13662 exec.cpp:131] Version: 0.19.0
      I0513 22:32:50.646881 13149 slave.cpp:1607] Got registration for executor '39652db7-efbf-4f2f-af8e-011e56546c67' of framework 20140513-223249-1740121354-38066-13118-0000
      I0513 22:32:50.648067 13149 slave.cpp:1726] Flushing queued task 39652db7-efbf-4f2f-af8e-011e56546c67 for executor '39652db7-efbf-4f2f-af8e-011e56546c67' of framework 20140513-223249-1740121354-38066-13118-0000
      I0513 22:32:50.648691 13150 cpushare.cpp:334] Updated 'cpu.shares' to 1024 (cpus 1) for container 69fd0ab4-ab01-4ec3-951f-0eab8a9fbe85
      I0513 22:32:50.648816 13145 mem.cpp:277] Updated 'memory.soft_limit_in_bytes' to 512MB for container 69fd0ab4-ab01-4ec3-951f-0eab8a9fbe85
      I0513 22:32:50.648823 13732 exec.cpp:205] Executor registered on slave 20140513-223249-1740121354-38066-13118-0
      Registered executor on smfd-bkq-03-sr4.devel.twitter.com
      Starting task 39652db7-efbf-4f2f-af8e-011e56546c67
      sh -c 'sleep 1000'
      Forked command at 13738
      I0513 22:32:50.656132 13135 slave.cpp:1962] Handling status update TASK_RUNNING (UUID: 358cda8b-b2a2-4b77-8ee5-9a5fc5889aa6) for task 39652db7-efbf-4f2f-af8e-011e56546c67 of framework 20140513-223249-1740121354-38066-13118-0000 from executor(1)@10.37.184.103:41570
      I0513 22:32:50.656723 13150 status_update_manager.cpp:320] Received status update TASK_RUNNING (UUID: 358cda8b-b2a2-4b77-8ee5-9a5fc5889aa6) for task 39652db7-efbf-4f2f-af8e-011e56546c67 of framework 20140513-223249-1740121354-38066-13118-0000
      I0513 22:32:50.657177 13150 status_update_manager.cpp:373] Forwarding status update TASK_RUNNING (UUID: 358cda8b-b2a2-4b77-8ee5-9a5fc5889aa6) for task 39652db7-efbf-4f2f-af8e-011e56546c67 of framework 20140513-223249-1740121354-38066-13118-0000 to master@10.37.184.103:38066
      I0513 22:32:50.657426 13150 slave.cpp:2089] Sending acknowledgement for status update TASK_RUNNING (UUID: 358cda8b-b2a2-4b77-8ee5-9a5fc5889aa6) for task 39652db7-efbf-4f2f-af8e-011e56546c67 of framework 20140513-223249-1740121354-38066-13118-0000 to executor(1)@10.37.184.103:41570
      I0513 22:32:50.657498 13145 master.cpp:2452] Status update TASK_RUNNING (UUID: 358cda8b-b2a2-4b77-8ee5-9a5fc5889aa6) for task 39652db7-efbf-4f2f-af8e-011e56546c67 of framework 20140513-223249-1740121354-38066-13118-0000 from slave 20140513-223249-1740121354-38066-13118-0 at slave(14)@10.37.184.103:38066 (smfd-bkq-03-sr4.devel.twitter.com)
      I0513 22:32:50.658360 13152 status_update_manager.cpp:398] Received status update acknowledgement (UUID: 358cda8b-b2a2-4b77-8ee5-9a5fc5889aa6) for task 39652db7-efbf-4f2f-af8e-011e56546c67 of framework 20140513-223249-1740121354-38066-13118-0000
      ../../src/tests/slave_recovery_tests.cpp:1076: Failure
      Failed to wait 10secs for update2
      /var/tmp/sclBXa2gl: line 8: 13118 Segmentation fault      './bin/mesos-tests.sh' '--gtest_filter=SlaveRecoveryTest/*' '--verbose' '--gtest_repeat=-1' '--gtest_break_on_failure'
      I0513 22:33:00.662961 13724 exec.cpp:458] Slave exited ... shutting down
      Shutting down
      Sending SIGTERM to process tree at pid 13738
      [bmahler@smfd-bkq-03-sr4 build]$ Killing the following process trees:
      [
      --- 13738 sleep 1000
      ]
      Command terminated with signal Terminated (pid: 13738)
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            bmahler Benjamin Mahler
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: