Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-4279

Docker executor truncates task's output when the task is killed.

    XMLWordPrintableJSON

    Details

    • Sprint:
      Mesosphere Sprint 35, Mesosphere Sprint 36
    • Story Points:
      5

      Description

      I'm implementing a graceful restarts of our mesos-marathon-docker setup and I came to a following issue:

      (it was already discussed on https://github.com/mesosphere/marathon/issues/2876 and guys form mesosphere got to a point that its probably a docker containerizer problem...)
      To sum it up:

      When i deploy simple python script to all mesos-slaves:

      #!/usr/bin/python
      
      from time import sleep
      import signal
      import sys
      import datetime
      
      def sigterm_handler(_signo, _stack_frame):
          print "got %i" % _signo
          print datetime.datetime.now().time()
          sys.stdout.flush()
          sleep(2)
          print datetime.datetime.now().time()
          print "ending"
          sys.stdout.flush()
          sys.exit(0)
      
      signal.signal(signal.SIGTERM, sigterm_handler)
      signal.signal(signal.SIGINT, sigterm_handler)
      
      try:
          print "Hello"
          i = 0
          while True:
              i += 1
              print datetime.datetime.now().time()
              print "Iteration #%i" % i
              sys.stdout.flush()
              sleep(1)
      finally:
          print "Goodbye"
      

      and I run it through Marathon like

      data = {
      	args: ["/tmp/script.py"],
      	instances: 1,
      	cpus: 0.1,
      	mem: 256,
      	id: "marathon-test-api"
      }
      

      During the app restart I get expected result - the task receives sigterm and dies peacefully (during my script-specified 2 seconds period)

      But when i wrap this python script in a docker:

      FROM node:4.2
      
      RUN mkdir /app
      ADD . /app
      WORKDIR /app
      ENTRYPOINT []
      

      and run appropriate application by Marathon:

      data = {
      	args: ["./script.py"],
      	container: {
      		type: "DOCKER",
      		docker: {
      			image: "bydga/marathon-test-api"
      		},
      		forcePullImage: yes
      	},
      	cpus: 0.1,
      	mem: 256,
      	instances: 1,
      	id: "marathon-test-api"
      }
      

      The task during restart (issued from marathon) dies immediately without having a chance to do any cleanup.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                bmahler Benjamin Mahler
                Reporter:
                bydga Martin Bydzovsky
                Shepherd:
                Jie Yu
              • Votes:
                9 Vote for this issue
                Watchers:
                23 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: