Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3611 Support Docker Containers In LinuxContainerExecutor
  3. YARN-7189

Container-executor doesn't remove Docker containers that error out early

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.9.0, 2.8.3, 3.0.1
    • 2.10.0, 2.9.2, 3.0.3
    • yarn
    • Reviewed

    Description

      Once the docker run command is executed, the docker container is created unless the return code is 125 meaning that the run command itself failed (https://docs.docker.com/engine/reference/run/#exit-status). Any error that happens after the docker run needs to remove the container during cleanup.

      container-executor.c:launch_docker_container_as_user
        snprintf(docker_command_with_binary, command_size, "%s %s", docker_binary, docker_command);
      
        fprintf(LOGFILE, "Launching docker container...\n");
        FILE* start_docker = popen(docker_command_with_binary, "r");
      

      This is fixed by YARN-5366, which changes how we remove containers. However, that was committed into 3.1.0. 2.8, 2.9, and 3.0 are all affected

      Attachments

        1. YARN-7189-branch-3.0.004.patch
          5 kB
          Eric Badger
        2. YARN-7189-branch-3.0.003.patch
          5 kB
          Eric Badger
        3. YARN-7189-branch-3.0.002.patch
          5 kB
          Eric Badger
        4. YARN-7189-branch-3.0.001.patch
          5 kB
          Eric Badger
        5. YARN-7189-b3.0.001.patch
          5 kB
          Eric Badger

        Activity

          People

            ebadger Eric Badger
            ebadger Eric Badger
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: