Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3611 Support Docker Containers In LinuxContainerExecutor
  3. YARN-6988

container-executor fails for docker when command length > 4096 B

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-beta1, 2.8.2
    • Component/s: yarn
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      run_docker and launch_docker_container_as_user allocate their command arrays using EXECUTOR_PATH_MAX, which is hardcoded to 4096 in configuration.h. Because of this, the full docker command can only be 4096 characters. If it is longer, it will be truncated and the command will fail with a parsing error. Because of the bind-mounting of volumes, the arguments to the docker command can quickly get large. For example, I passed the 4096 limit with an 11 disk node.

        Attachments

        1. YARN-6988-branch-2.8.002.patch
          6 kB
          Eric Badger
        2. YARN-6988-branch-2.002.patch
          6 kB
          Eric Badger
        3. YARN-6988.002.patch
          6 kB
          Eric Badger
        4. YARN-6988.001.patch
          5 kB
          Eric Badger

          Issue Links

            Activity

              People

              • Assignee:
                ebadger Eric Badger
                Reporter:
                ebadger Eric Badger
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: