Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3611 Support Docker Containers In LinuxContainerExecutor
  3. YARN-8384

stdout.txt, stderr.txt logs of a launched docker container is coming with primary group of submit user instead of hadoop

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      When yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users is set to true, and yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user is set to nobody.

      This will cause the docker to run as nobody:nobody in yarn mode.
      The log files will be initialized as nobody:nobody:

      rw-rr- 1 nobody hadoop 354 May 31 17:33 container-localizer-syslog
      rw-rr- 1 nobody hadoop 1042 May 31 17:35 directory.info
      rw-r---- 1 nobody hadoop 4944 May 31 17:35 launch_container.sh
      rw-rr- 1 nobody hadoop 440 May 31 17:35 prelaunch.err
      rw-rr- 1 nobody hadoop 100 May 31 17:35 prelaunch.out
      rw-r---- 1 nobody nobody 18733 May 31 17:37 stderr.txt
      rw-r---- 1 nobody nobody 400 May 31 17:35 stdout.txt
      

      This causes YARN NM cannot read stderr.txt and stdout.txt

        Attachments

          Activity

            People

            • Assignee:
              eyang Eric Yang
              Reporter:
              sunilg Sunil Govindan
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: