Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15096

start-build-env.sh can create a docker image that fills up disk

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.1.0
    • None
    • build
    • None

    Description

      start-build-env.sh has the potential to build an image that can fill up root disks by exploding a sparse file.

      In my case, the right ingredients are:
      Ubuntu 17.04
      Docker 17.09.0
      AUFS storage driver
      userId and groupid with a high number

      This happens when building the hadoop-build-${USER_ID} image, specifically in the

      RUN useradd -g ${GROUP_ID} -u ${USER_ID} -k /root -m ${USER_NAME}
      

      command.

      The reason for this:
      /var/log/lastlog is a sparse file that pre-reserves based on highest seen UID and GID, in my case, those numbers are very high (above 1 billion). Locally, this result in a sparse file that reports as 443 GB. However, under docker and specifically AUFS, it appears that his file isn't sparse and it tries to allocate the whole file.

      If you start this script and walk away to wait for it to finish, you come back to a computer with a completely full disk.

      Luckily, the fix is quite easy, simply add the `-l` option to useradd which won't create those files

      Attachments

        Activity

          People

            Unassigned Unassigned
            addisonj@gmail.com Addison Higham
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: