Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14026

start-build-env.sh: invalid docker image name

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4, 2.8.2
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      start-build-env.sh using the current user name to generate a docker image name. But the current user name can contains some not english characters and upper letters (after all this is usually the name/nickname of the owner). Both of them are not supported in docker image names, so the script will fail.

      1. HADOOP-14026_v2.patch
        0.7 kB
        Gergő Pásztor
      2. HADOOP-14026_v1.patch
        1 kB
        Gergő Pásztor

        Activity

        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - 2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11350 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11350/)
        HADOOP-14026. start-build-env.sh: invalid docker image name (Contributed (templedf: rev c8bd8ac7f9642b8db5949a914f209d5de3d4ef9b)

        • (edit) start-build-env.sh
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11350 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11350/ ) HADOOP-14026 . start-build-env.sh: invalid docker image name (Contributed (templedf: rev c8bd8ac7f9642b8db5949a914f209d5de3d4ef9b) (edit) start-build-env.sh
        Hide
        templedf Daniel Templeton added a comment -

        Committed to branch-2.8, branch-2, and trunk. Thanks, Gergő Pásztor, for the patch, and Allen Wittenauer for the reviews.

        Show
        templedf Daniel Templeton added a comment - Committed to branch-2.8, branch-2, and trunk. Thanks, Gergő Pásztor , for the patch, and Allen Wittenauer for the reviews.
        Hide
        templedf Daniel Templeton added a comment -

        LGTM. +1. I'll wait a little while to commit to give Allen Wittenauer a chance to respond.

        Show
        templedf Daniel Templeton added a comment - LGTM. +1. I'll wait a little while to commit to give Allen Wittenauer a chance to respond.
        Hide
        pairg Gergő Pásztor added a comment -

        Allen Wittenauer Please review it!

        Show
        pairg Gergő Pásztor added a comment - Allen Wittenauer Please review it!
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 13m 33s trunk passed
        +1 mvnsite 9m 6s trunk passed
        +1 mvnsite 9m 8s the patch passed
        +1 shellcheck 0m 11s There were no new shellcheck issues.
        +1 shelldocs 0m 8s There were no new shelldocs issues.
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 unit 14m 49s root in the patch passed.
        +1 asflicense 0m 36s The patch does not generate ASF License warnings.
        48m 13s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HADOOP-14026
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12850024/HADOOP-14026_v2.patch
        Optional Tests asflicense mvnsite unit shellcheck shelldocs
        uname Linux 02e9b3f9539b 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 312b36d
        shellcheck v0.4.5
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11532/testReport/
        modules C: . U: .
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11532/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 13m 33s trunk passed +1 mvnsite 9m 6s trunk passed +1 mvnsite 9m 8s the patch passed +1 shellcheck 0m 11s There were no new shellcheck issues. +1 shelldocs 0m 8s There were no new shelldocs issues. +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 14m 49s root in the patch passed. +1 asflicense 0m 36s The patch does not generate ASF License warnings. 48m 13s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-14026 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12850024/HADOOP-14026_v2.patch Optional Tests asflicense mvnsite unit shellcheck shelldocs uname Linux 02e9b3f9539b 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 312b36d shellcheck v0.4.5 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11532/testReport/ modules C: . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11532/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        pairg Gergő Pásztor added a comment -

        Attached.

        Show
        pairg Gergő Pásztor added a comment - Attached.
        Hide
        pairg Gergő Pásztor added a comment -

        Good idea!

        Show
        pairg Gergő Pásztor added a comment - Good idea!
        Hide
        aw Allen Wittenauer added a comment - - edited

        Rather than go through all that work, why not just use the uid instead of the username for the docker name?

        ie.

         docker build -t "hadoop-build-$(id -u)" - <<UserSpecificDocker
        
        Show
        aw Allen Wittenauer added a comment - - edited Rather than go through all that work, why not just use the uid instead of the username for the docker name? ie. docker build -t "hadoop-build-$(id -u)" - <<UserSpecificDocker
        Hide
        pairg Gergő Pásztor added a comment - - edited

        Good to know that some Linux are ignoring/extending the POSIX standard.
        I think it should be better to support every OS and prevent as much failures as we can. If we think for this image name postfix as an input from the environment it's okey to validate the value and ask for another input if it's invalid. But if you disagree with me I can modify the patch.

        Show
        pairg Gergő Pásztor added a comment - - edited Good to know that some Linux are ignoring/extending the POSIX standard. I think it should be better to support every OS and prevent as much failures as we can. If we think for this image name postfix as an input from the environment it's okey to validate the value and ask for another input if it's invalid. But if you disagree with me I can modify the patch.
        Hide
        aw Allen Wittenauer added a comment -

        not contains any non-english characters

        That's a violation of POSIX: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_437 which references http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_278 .

        nor any upper case english characters

        it's easier then to just pass it through tr.

        Show
        aw Allen Wittenauer added a comment - not contains any non-english characters That's a violation of POSIX: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_437 which references http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_278 . nor any upper case english characters it's easier then to just pass it through tr.
        Hide
        pairg Gergő Pásztor added a comment -

        Allen Wittenauer Yeah, but we can't assume that somebody's login name not contains any non-english characters nor any upper case english characters. Docker's rule for image names is much more strict than a usual OS's rule for login names. Check out this for example: http://stackoverflow.com/a/25388074

        I spent some hours to debug what's the problem with the script, because the error message was misleading. I found out that my user name contains an upper case english character. I can imagine that I'm not the only one who fell into this trap.

        Show
        pairg Gergő Pásztor added a comment - Allen Wittenauer Yeah, but we can't assume that somebody's login name not contains any non-english characters nor any upper case english characters. Docker's rule for image names is much more strict than a usual OS's rule for login names. Check out this for example: http://stackoverflow.com/a/25388074 I spent some hours to debug what's the problem with the script, because the error message was misleading. I found out that my user name contains an upper case english character. I can imagine that I'm not the only one who fell into this trap.
        Hide
        aw Allen Wittenauer added a comment -

        the current user name can contains some not english characters and upper letters (after all this is usually the name/nickname of the owner)

        http://unix.stackexchange.com/questions/76354/who-sets-user-and-username-environment-variables is a quick link to provide more info, but the sum of it is that $USER is almost always set to the user's login name via the login(1) process. I honestly can't remember the last time I saw a system that didn't do that. Now there are cases where $USER isn't set (e.g., didn't go in through login), and the appropriate response is to set $USER to the name (e.g., output of id -u -n).

        What is your environment that this is not true?

        Show
        aw Allen Wittenauer added a comment - the current user name can contains some not english characters and upper letters (after all this is usually the name/nickname of the owner) http://unix.stackexchange.com/questions/76354/who-sets-user-and-username-environment-variables is a quick link to provide more info, but the sum of it is that $USER is almost always set to the user's login name via the login(1) process. I honestly can't remember the last time I saw a system that didn't do that. Now there are cases where $USER isn't set (e.g., didn't go in through login), and the appropriate response is to set $USER to the name (e.g., output of id -u -n). What is your environment that this is not true?

          People

          • Assignee:
            pairg Gergő Pásztor
            Reporter:
            pairg Gergő Pásztor
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development