Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-6132

Mesos 1.0.1 marathon 1.1.1 deployment fails no such image, container or task

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 1.0.1
    • 1.0.1
    • None
    • None

    Description

      Trying to launch docker containers with marathon on bridged network but they fail. I have tried to pull containers manually but it did not work too.
      Here are some logs from Mesos Salve/Master and Marathon.

      Mesos Slave Logs:

      "sudo ./bin/mesos-slave.sh --ip=<IP OF SLAVE> --master=zk://<IP OF ZOOKEEPER>:2181/mesos --advertise_ip=<IP OF SLAVE> --docker_socket=/var/run/docker.sock --work_dir=$/tmp/mesos-slave --containerizers=docker --executor_registration_timeout=5mins --isolation=docker/runtime"

      I0907 06:02:58.206800 5820 slave.cpp:1495] Got assigned task helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545 for framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 [23/9246]
      I0907 06:02:58.208921 5820 slave.cpp:1614] Launching task helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545 for framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 [22/9246]
      I0907 06:02:58.210595 5820 paths.cpp:528] Trying to chown '$/tmp/mesos-slave/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185ee[21/9246]
      xecutors/helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545/runs/0a52cc6d-499d-487d-983a-d3d8e62213e2' to user 'aio' [20/9246]
      I0907 06:02:58.214004 5820 slave.cpp:5674] Launching executor helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 with[19/9246]
      s cpus:0.1; mem:32 in work directory '$/tmp/mesos-slave/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000/executor[18/9246]
      rld.43c00fb5-74e2-11e6-80f0-a2e45809a545/runs/0a52cc6d-499d-487d-983a-d3d8e62213e2' [17/9246]
      I0907 06:02:58.216660 5820 slave.cpp:1840] Queuing task 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' for executor 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' [16/9246]
      ork d034dd41-d32c-4e81-a168-070c2185eefe-0000 [15/9246]
      I0907 06:02:58.217999 5817 docker.cpp:1042] Starting container '0a52cc6d-499d-487d-983a-d3d8e62213e2' for task 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' (and exe[14/9246]
      I0907 06:02:58.217999 5817 docker.cpp:1042] Starting container '0a52cc6d-499d-487d-983a-d3d8e62213e2' for task 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' (and exe[14/9246$
      lloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545') of framework 'd034dd41-d32c-4e81-a168-070c2185eefe-0000'
      I0907 06:03:02.005821 5820 docker.cpp:658] Checkpointing pid 5853 to '$/tmp/mesos-slave/meta/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-07$
      c2185eefe-0000/executors/helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545/runs/0a52cc6d-499d-487d-983a-d3d8e62213e2/pids/forked.pid'
      I0907 06:03:02.057451 5814 slave.cpp:2828] Got registration for executor 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000
      from executor(1)@192.168.50.14:59541
      I0907 06:03:02.059079 5814 docker.cpp:1443] Ignoring updating container '0a52cc6d-499d-487d-983a-d3d8e62213e2' with resources passed to update is identical to existing resources
      I0907 06:03:02.059430 5814 slave.cpp:2005] Sending queued task 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' to executor 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' of
      framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 at executor(1)@192.168.50.14:59541
      I0907 06:03:07.500459 5815 slave.cpp:3211] Handling status update TASK_FAILED (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a5$
      5 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 from executor(1)@192.168.50.14:59541
      E0907 06:03:07.539857 5820 slave.cpp:3456] Failed to update resources for container 0a52cc6d-499d-487d-983a-d3d8e62213e2 of executor 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809$
      545' running task helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545 on status update for terminal task, destroying container: Failed to run 'docker -H unix:///var/run/docker.sock i$
      spect mesos-9b5505a5-71d9-47cd-a810-898bb24be347-S0.0a52cc6d-499d-487d-983a-d3d8e62213e2': exited with status 1; stderr='Error: No such image, container or task: mesos-9b5505a5-7$d9-47cd-a810-898bb24be347-S0.0a52cc6d-499d-487d-983a-d3d8e62213e2
      '
      I0907 06:03:07.540410 5820 docker.cpp:1852] Destroying container '0a52cc6d-499d-487d-983a-d3d8e62213e2'
      I0907 06:03:07.540549 5820 docker.cpp:1938] Sending SIGTERM to executor with pid: 5853
      I0907 06:03:07.540467 5814 status_update_manager.cpp:320] Received status update TASK_FAILED (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c00fb5-74e2-11e6-$0f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000
      I0907 06:03:07.543401 5814 status_update_manager.cpp:825] Checkpointing UPDATE for status update TASK_FAILED (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c$0fb5-74e2-11e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000
      I0907 06:03:07.555461 5820 docker.cpp:1980] Running docker stop on container '0a52cc6d-499d-487d-983a-d3d8e62213e2'
      I0907 06:03:07.557958 5816 slave.cpp:3732] executor(1)@192.168.50.14:59541 exited
      I0907 06:03:07.642285 5821 docker.cpp:2132] Executor for container '0a52cc6d-499d-487d-983a-d3d8e62213e2' has exited
      I0907 06:03:07.644544 5821 slave.cpp:4089] Executor 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 terminated with sign$l Terminated
      I0907 06:03:07.703641 5818 slave.cpp:3604] Forwarding the update TASK_FAILED (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a54$
      of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 to master@192.168.50.10:5050
      I0907 06:03:07.704087 5818 slave.cpp:3514] Sending acknowledgement for status update TASK_FAILED (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c00fb5-74e2-1$
      e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 to executor(1)@192.168.50.14:59541
      E0907 06:03:07.704627 5822 process.cpp:2105] Failed to shutdown socket with fd 11: Transport endpoint is not connected
      I0907 06:03:07.743113 5814 status_update_manager.cpp:392] Received status update acknowledgement (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c00fb5-74e2-1$
      e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000
      I0907 06:03:07.743348 5814 status_update_manager.cpp:825] Checkpointing ACK for status update TASK_FAILED (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c00f$
      5-74e2-11e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000
      I0907 06:03:07.804318 5814 slave.cpp:4193] Cleaning up executor 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 at execu$
      or(1)@192.168.50.14:59541
      I0907 06:03:07.805060 5814 slave.cpp:4281] Cleaning up framework d034dd41-d32c-4e81-a168-070c2185eefe-0000
      I0907 06:03:07.805099 5819 gc.cpp:55] Scheduling '$/tmp/mesos-slave/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000/executors$
      helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545/runs/0a52cc6d-499d-487d-983a-d3d8e62213e2' for gc 6.99999068706074days in the future
      I0907 06:03:07.805243 5819 gc.cpp:55] Scheduling '$/tmp/mesos-slave/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000/executors$
      helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' for gc 6.99999068475852days in the future
      I0907 06:03:07.805344 5819 gc.cpp:55] Scheduling '$/tmp/mesos-slave/meta/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000/exec$
      tors/helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545/runs/0a52cc6d-499d-487d-983a-d3d8e62213e2' for gc 6.99999068390222days in the future
      I0907 06:03:07.805438 5819 gc.cpp:55] Scheduling '$/tmp/mesos-slave/meta/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000/exec$
      tors/helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' for gc 6.99999068312593days in the future
      I0907 06:03:07.805673 5820 status_update_manager.cpp:282] Closing status update streams for framework d034dd41-d32c-4e81-a168-070c2185eefe-0000
      I0907 06:03:07.805923 5817 gc.cpp:55] Scheduling '$/tmp/mesos-slave/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000' for gc 6$
      99999067363259days in the future
      I0907 06:03:07.806212 5818 gc.cpp:55] Scheduling '$/tmp/mesos-slave/meta/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000' for
      gc 6.99999066995259days in the future

      Mesos Master Logs:

      "./bin/mesos-master.sh --advertise_ip=192.168.50.10 --zk=zk://192.168.50.10:2181/mesos --port=5050 --quorum=1 --registry=in_memory --work_dir=$/tmp/mesos_master"

      I0907 06:06:39.715235 419 master.cpp:4046] Processing REVIVE call for framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e88
      5e56e@127.0.1.1:41351
      I0907 06:06:39.715349 419 hierarchical.cpp:1022] Removed offer filters for framework d034dd41-d32c-4e81-a168-070c2185eefe-0000
      I0907 06:06:39.716269 419 master.cpp:5725] Sending 1 offers to framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@1
      27.0.1.1:41351
      I0907 06:06:39.770603 416 master.cpp:3342] Processing ACCEPT call for offers: [ 9b5505a5-71d9-47cd-a810-898bb24be347-O54 ] on agent 9b5505a5-71d9-47cd-a810-898bb24be347-S0 at sl
      ave(1)@192.168.50.14:5051 (edge2.plumgrid.com) for framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@127.0.1.1:41351
      I0907 06:06:39.772132 416 master.cpp:7447] Adding task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809a545 with resources cpus:1; mem:128 on agent 9b5505a5-71d9-47cd-a810-898b
      b24be347-S0 (edge2.plumgrid.com)
      I0907 06:06:39.772290 416 master.cpp:3831] Launching task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at sc
      heduler-91481722-8a3f-4297-97b7-a8c5e885e56e@127.0.1.1:41351 with resources cpus:1; mem:128 on agent 9b5505a5-71d9-47cd-a810-898bb24be347-S0 at slave(1)@192.168.50.14:5051 (
      edge2.plumgrid.com)
      I0907 06:06:40.471755 418 master.cpp:5725] Sending 1 offers to framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@1
      27.0.1.1:41351
      I0907 06:06:40.475045 415 master.cpp:3951] Processing DECLINE call for offers: [ 9b5505a5-71d9-47cd-a810-898bb24be347-O55 ] for framework d034dd41-d32c-4e81-a168-070c2185eefe-00
      00 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@127.0.1.1:41351
      I0907 06:06:47.039434 420 http.cpp:381] HTTP GET for /master/state from 192.168.50.1:33520 with User-Agent='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (K
      HTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36'
      I0907 06:06:48.750298 418 master.cpp:5137] Status update TASK_FAILED (UUID: 7d401455-72c5-42a0-90b6-40ab849f6075) for task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809a545 of fra
      mework d034dd41-d32c-4e81-a168-070c2185eefe-0000 from agent 9b5505a5-71d9-47cd-a810-898bb24be347-S0 at slave(1)@192.168.50.14:5051 (edge2.plumgrid.com)
      I0907 06:06:48.750366 418 master.cpp:5199] Forwarding status update TASK_FAILED (UUID: 7d401455-72c5-42a0-90b6-40ab849f6075) for task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809
      a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000
      I0907 06:06:48.750640 418 master.cpp:6841] Updating the state of task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (lat
      est state: TASK_FAILED, status update state: TASK_FAILED)
      I0907 06:06:48.811859 419 master.cpp:4262] Processing ACKNOWLEDGE call 7d401455-72c5-42a0-90b6-40ab849f6075 for task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809a545 of framework
      d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@127.0.1.1:41351 on agent 9b5505a5-71d9-47cd-a810-898bb24be347-S0
      I0907 06:06:48.811954 419 master.cpp:6907] Removing task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809a545 with resources cpus:1; mem:128 of framework d034dd41-d32c-4e81-a16
      8-070c2185eefe-0000 on agent 9b5505a5-71d9-47cd-a810-898bb24be347-S0 at slave(1)@192.168.50.14:5051 (edge2.plumgrid.com)
      I0907 06:06:49.482637 415 master.cpp:5725] Sending 1 offers to framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@1
      27.0.1.1:41351
      I0907 06:06:49.485888 421 master.cpp:3951] Processing DECLINE call for offers: [ 9b5505a5-71d9-47cd-a810-898bb24be347-O56 ] for framework d034dd41-d32c-4e81-a168-070c2185eefe-00
      00 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@127.0.1.1:41351

      Marathon Logs:

      "MESOS_NATIVE_JAVA_LIBRARY=/home/aio/mesos-1.0.1/build/src/.libs/libmesos.so bin/start --master zk://192.168.50.10:2181/mesos --zk zk://192.168.50.10:2181/marathon --event_subscriber http_callback --task_launch_timeout 300000"

      [2016-09-07 06:11:42,148] INFO [/helloworld]: new app detected (mesosphere.marathon.upgrade.GroupVersioningUtil$:ForkJoinPool-2-worker-13) [141/10000]
      [2016-09-07 06:11:42,150] INFO Computed new deployment plan: [140/10000]
      DeploymentPlan 2016-09-07T10:11:42.148Z [139/10000]
      step 1: [138/10000]

      • Start(App(/helloworld, image="tutum/hello-world")), instances=0) [137/10000]
        step 2: [136/10000]
      • Scale(App(/helloworld, image="tutum/hello-world")), instances=1) [135/10000]
        (mesosphere.marathon.state.GroupManager:ForkJoinPool-2-worker-13) [134/10000]
        [2016-09-07 06:11:42,150] INFO Deploy plan with force=false: [133/10000]
        DeploymentPlan 2016-09-07T10:11:42.148Z [132/10000]
        step 1: [131/10000]
      • Start(App(/helloworld, image="tutum/hello-world")), instances=0) [130/10000]
        step 2: [129/10000]
      • Scale(App(/helloworld, image="tutum/hello-world")), instances=1) [128/10000]
        (mesosphere.marathon.MarathonSchedulerService$$EnhancerByGuice$$7b7c577c:ForkJoinPool-2-worker-13) [127/10000]
        [2016-09-07 06:11:42,150] INFO [/helloworld] storing new app version 2016-09-07T10:11:42.148Z (mesosphere.marathon.state.GroupManager:ForkJoinPool-2-worker-13) [126/10000]
        [2016-09-07 06:11:42,217] INFO Starting app /helloworld (mesosphere.marathon.SchedulerActions:marathon-akka.actor.default-dispatcher-30) [125/10000]
        [2016-09-07 06:11:42,217] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-53) [124/10000]
        [2016-09-07 06:11:42,217] INFO Already running 0 instances of /helloworld. Not scaling. (mesosphere.marathon.SchedulerActions:marathon-akka.actor.default-dispatcher-30)[123/10000]
        [2016-09-07 06:11:42,217] INFO Successfully started 0 instances of /helloworld (mesosphere.marathon.upgrade.AppStartActor:marathon-akka.actor.default-dispatcher-55) [122/10000]
        [2016-09-07 06:11:42,217] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-41) [121/10000]
        [2016-09-07 06:11:42,217] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-30) [120/10000]
        [2016-09-07 06:11:42,220] INFO Started appTaskLaunchActor for /helloworld version 2016-09-07T10:11:42.148Z with initial count 1 (mesosphere.marathon.core.launchqueue.im[119/10000]
        auncherActor:marathon-akka.actor.default-dispatcher-56) [118/10000]
        [2016-09-07 06:11:42,220] INFO activating matcher ActorOfferMatcher(Actorakka://marathon/user/launchQueue/1/2-helloworld#-434864973). (mesosphere.marathon.core.matche[117/10000]
        mpl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-18) [116/10000]
        [2016-09-07 06:11:42,220] INFO Received offers WANTED notification (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-18) [115/10000]
        [2016-09-07 06:11:42,220] INFO => revive offers NOW, canceling any scheduled revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-d[114/10000]
        8) [113/10000]
        [2016-09-07 06:11:42,220] INFO 2 further revives still needed. Repeating reviveOffers according to --revive_offers_repetitions 3 (mesosphere.marathon.core.flow.impl.Rev[112/10000]
        tor:marathon-akka.actor.default-dispatcher-18) [111/10000]
        [2016-09-07 06:11:42,221] INFO => Schedule next revive at 2016-09-07T10:11:47.220Z in 5000 milliseconds, adhering to --min_revive_offers_interval 5000 (ms) (mesosphere.[110/10000]
        re.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-45) [109/10000]
        [2016-09-07 06:11:42,227] INFO Processing LaunchEphemeral(LaunchedEphemeral(task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a545],AgentInfo(edge2.plumgrid.com,Some(9b5[108/10000]
        47cd-a810-898bb24be347-S0),Buffer()),2016-09-07T10:11:42.148Z,Status(2016-09-07T10:11:42.226Z,None,None),List())) for task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a[107/10000]
        phere.marathon.core.launcher.impl.OfferProcessorImpl:ForkJoinPool-2-worker-13) [106/10000]
        [2016-09-07 06:11:42,227] INFO Request Launch for task 'helloworld.794398bd-74e3-11e6-80f0-a2e45809a545', version '2016-09-07T10:11:42.148Z'. 1 tasksToLaunch, 0 in flig[105/10000]
        rmed. not backing off (mesosphere.marathon.core.launchqueue.impl.AppTaskLauncherActor:marathon-akka.actor.default-dispatcher-56) [104/10000]
        [2016-09-07 06:11:42,227] INFO No tasks left to launch. Stop receiving offers for /helloworld, 2016-09-07T10:11:42.148Z (mesosphere.marathon.core.launchqueue.impl.AppTa[103/10000]
        ctor:marathon-akka.actor.default-dispatcher-56) [102/10000]
        [2016-09-07 06:11:42,227] INFO removing matcher ActorOfferMatcher(Actorakka://marathon/user/launchQueue/1/2-helloworld#-434864973) (mesosphere.marathon.core.matcher.m[101/10000]
        .OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-56) [100/10000]
        [2016-09-07 06:11:42,228] INFO Received offers NOT WANTED notification, canceling 2 revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.def[99/10000]
        tcher-56) [98/10000]
        [2016-09-07 06:11:42,228] INFO Finished processing 9b5505a5-71d9-47cd-a810-898bb24be347-O126. Matched 1 ops after 2 passes. cpus 3.0; mem 6832.0; disk 79158.0; [97/10000]
        1000->32000 left. (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-56) [96/10000]
        [2016-09-07 06:11:42,253] INFO receiveTaskUpdate: updating status of task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a545] (mesosphere.marathon.core.launchqueue.impl.Ap[95/10000]
        herActor:marathon-akka.actor.default-dispatcher-18) [94/10000]
        [2016-09-07 06:11:42,255] INFO Task launch for 'task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a545]' was accepted. 0 tasksToLaunch, 0 in flight, 1 confirmed. not bac[93/10000]
        mesosphere.marathon.core.launchqueue.impl.AppTaskLauncherActor:marathon-akka.actor.default-dispatcher-55) [92/10000]
        [2016-09-07 06:11:42,277] INFO Updated groups/apps according to deployment plan fb9a07c1-c317-4bcf-8e59-d1783683fa71 (mesosphere.marathon.state.GroupManager:ForkJoinPool[91/10000]
        3) [90/10000]
        [2016-09-07 06:11:42,277] INFO Deployment acknowledged. Waiting to get processed: [89/10000]
        DeploymentPlan 2016-09-07T10:11:42.148Z [88/10000]
        step 1: [87/10000]
      • Start(App(/helloworld, image="tutum/hello-world")), instances=0) [86/10000]
        step 2: [85/10000]
      • Scale(App(/helloworld, image="tutum/hello-world")), instances=1) [84/10000]
        (mesosphere.marathon.state.GroupManager:ForkJoinPool-2-worker-3) [83/10000]
        [2016-09-07 06:11:42,277] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-55) [82/10000]
        [2016-09-07 06:11:42,277] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-18) [81/10000]
        [2016-09-07 06:11:42,278] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "POST //192.168.50.10:8080/v2/apps HTTP/1.1" 201 880 "http://192.168.50.10:8080/ui/" "Mozill[80/10000]
        intosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" 13 (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a[79/10000]
        1010311355-37) [78/10000]
        [2016-09-07 06:11:42,320] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "GET //192.168.50.10:8080/ui/img/icons/icon-small-app.svg HTTP/1.1" 200 1519 "http://192.168[77/10000]
        0/ui/main.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosR[76/10000]
        $EnhancerByGuice$$7a66064c:qtp1010311355-36) [75/10000]
        [2016-09-07 06:11:42,320] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "GET //192.168.50.10:8080/ui/img/icons/icon-mini-ellipse-white.svg HTTP/1.1" 200 976 "http:/[74/10000]
        0.10:8080/ui/main.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.htt[73/10000]
        uestLog$$EnhancerByGuice$$7a66064c:qtp1010311355-40) [72/10000]
        [2016-09-07 06:11:42,277] INFO Updated groups/apps according to deployment plan fb9a07c1-c317-4bcf-8e59-d1783683fa71 (mesosphere.marathon.state.GroupManager:ForkJoinPool[91/10000$
        3)
        [2016-09-07 06:11:42,277] INFO Deployment acknowledged. Waiting to get processed:
        DeploymentPlan 2016-09-07T10:11:42.148Z
        step 1:
      • Start(App(/helloworld, image="tutum/hello-world")), instances=0)
        step 2:
      • Scale(App(/helloworld, image="tutum/hello-world")), instances=1)
        (mesosphere.marathon.state.GroupManager:ForkJoinPool-2-worker-3)
        [2016-09-07 06:11:42,277] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-55)
        [2016-09-07 06:11:42,277] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-18)
        [2016-09-07 06:11:42,278] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "POST //192.168.50.10:8080/v2/apps HTTP/1.1" 201 880 "http://192.168.50.10:8080/ui/" "Mozilla/5.0 (Mac
        intosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" 13 (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064c:qtp
        1010311355-37)
        [2016-09-07 06:11:42,320] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "GET //192.168.50.10:8080/ui/img/icons/icon-small-app.svg HTTP/1.1" 200 1519 "http://192.168.50.10:808
        0/ui/main.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$
        $EnhancerByGuice$$7a66064c:qtp1010311355-36)
        [2016-09-07 06:11:42,320] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "GET //192.168.50.10:8080/ui/img/icons/icon-mini-ellipse-white.svg HTTP/1.1" 200 976 "http://192.168.5
        0.10:8080/ui/main.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosReq
        uestLog$$EnhancerByGuice$$7a66064c:qtp1010311355-40)
        [2016-09-07 06:11:42,320] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "GET //192.168.50.10:8080/ui/img/icons/icon-mini-deploying-color.png HTTP/1.1" 200 1144 "http://192.16
        8.50.10:8080/ui/main.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.Chao$
        RequestLog$$EnhancerByGuice$$7a66064c:qtp1010311355-34)
        [2016-09-07 06:11:46,053] INFO 192.168.50.1 - - [07/Sep/2016:10:11:46 +0000] "GET //192.168.50.10:8080/v2/deployments HTTP/1.1" 200 376 "http://192.168.50.10:8080/ui/" "Mozilla/5$
        0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064$
        :qtp1010311355-68)
        [2016-09-07 06:11:46,053] INFO 192.168.50.1 - - [07/Sep/2016:10:11:46 +0000] "GET //192.168.50.10:8080/v2/queue HTTP/1.1" 200 12 "http://192.168.50.10:8080/ui/" "Mozilla/5.0 (Mac$
        ntosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064c:qtp10$
        0311355-66)
        [2016-09-07 06:11:46,055] INFO 192.168.50.1 - - [07/Sep/2016:10:11:46 +0000] "GET //192.168.50.10:8080/v2/groups?embed=group.groups&embed=group.apps&embed=group.apps.deployments&$
        mbed=group.apps.counts&embed=group.apps.readiness HTTP/1.1" 200 1090 "http://192.168.50.10:8080/ui/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, l$
        ke Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064c:qtp1010311355-33)
        [2016-09-07 06:11:51,052] INFO 192.168.50.1 - - [07/Sep/2016:10:11:51 +0000] "GET //192.168.50.10:8080/v2/deployments HTTP/1.1" 200 376 "http://192.168.50.10:8080/ui/" "Mozilla/5$
        0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064$
        :qtp1010311355-40)
        [2016-09-07 06:11:51,053] INFO 192.168.50.1 - - [07/Sep/2016:10:11:51 +0000] "GET //192.168.50.10:8080/v2/queue HTTP/1.1" 200 12 "http://192.168.50.10:8080/ui/" "Mozilla/5.0 (Mac$
        ntosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064c:qtp10$
        0311355-37)
        [2016-09-07 06:11:51,058] INFO 192.168.50.1 - - [07/Sep/2016:10:11:51 +0000] "GET //192.168.50.10:8080/v2/groups?embed=group.groups&embed=group.apps&embed=group.apps.deployments&$
        mbed=group.apps.counts&embed=group.apps.readiness HTTP/1.1" 200 1090 "http://192.168.50.10:8080/ui/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, l$
        ke Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064c:qtp1010311355-36)
        [2016-09-07 06:11:51,492] INFO Received status update for task helloworld.794398bd-74e3-11e6-80f0-a2e45809a545: TASK_FAILED (Container exited with status 125) (mesosphere.maratho$
        .MarathonScheduler$$EnhancerByGuice$$3a82e0a2:Thread-189)
        [2016-09-07 06:11:51,509] INFO Removed app [/helloworld] from tracker (mesosphere.marathon.core.task.tracker.TaskTracker$TasksByApp$:marathon-akka.actor.default-dispatcher-53)
        [2016-09-07 06:11:51,510] INFO Increasing delay. Task launch delay for [/helloworld] changed from [0 milliseconds] to [1 seconds]. (mesosphere.marathon.core.launchqueue.impl.Rate$
        imiter$:marathon-akka.actor.default-dispatcher-55)
        [2016-09-07 06:11:51,510] INFO receiveTaskUpdate: task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a545] finished (mesosphere.marathon.core.launchqueue.impl.AppTaskLauncherActor:$
        arathon-akka.actor.default-dispatcher-53)
        [2016-09-07 06:11:51,511] INFO Sending event notification for task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a545] of app [/helloworld]: TASK_FAILED (mesosphere.marathon.core.t$
        sk.update.impl.steps.PostToEventStreamStepImpl$$EnhancerByGuice$$8b5bf639:marathon-akka.actor.default-dispatcher-52)
        [2016-09-07 06:11:51,511] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-30)
        [2016-09-07 06:11:51,512] WARN New task [task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a545]] failed during app /helloworld scaling, queueing another task (mesosphere.marathon$
        upgrade.TaskStartActor:marathon-akka.actor.default-dispatcher-30)

      Following JSON was used to launch containers:

      {
      "id": "helloworld",
      "cpus": 1,
      "mem": 128,
      "instances": 1,
      "ipAddress": {},
      "container": {
      "type": "DOCKER",
      "docker":

      { "image": "tutum/hello-world", "forcePullImage": true, "network": "BRIDGE" }

      }
      }

      Attachments

        Activity

          People

            Unassigned Unassigned
            arslanqadeer Arslan Qadeer
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: