Hadoop Common
  1. Hadoop Common
  2. HADOOP-7603

Set default hdfs, mapred uid, and hadoop group gid for RPM packages

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0, 0.23.0
    • Fix Version/s: 0.20.205.0, 0.23.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Java, Redhat EL, Ubuntu

    • Release Note:
      Set hdfs uid, mapred uid, and hadoop gid to fixed numbers (201, 202, and 123, respectively).

      Description

      Hadoop rpm package creates hdfs, mapped users, and hadoop group for automatically setting up pid directory and log directory with proper permission. The default headless users should have a fixed uid, and gid numbers defined.

      Searched through the standard uid and gid on both Redhat and Debian distro. It looks like:

      uid: 201 for hdfs
      uid: 202 for mapred
      gid: 49 for hadoop
      

      would be free for use.

      1. HADOOP-7603-trunk.patch
        5 kB
        Eric Yang
      2. HADOOP-7603.patch
        2 kB
        Eric Yang

        Activity

        Hide
        Allen Wittenauer added a comment -

        49 is not a safe gid on RHEL6.

        Also, what happens when these uid/gids get used by something else?

        Show
        Allen Wittenauer added a comment - 49 is not a safe gid on RHEL6. Also, what happens when these uid/gids get used by something else?
        Hide
        Eli Collins added a comment -

        What if these users already exist on the system?

        Show
        Eli Collins added a comment - What if these users already exist on the system?
        Hide
        Eric Yang added a comment -

        49 is not a safe gid on RHEL6.

        Base on Redhat 6 document, it should be available. What group uses 49?

        Reference:

        http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/s1-users-groups-standard-groups.html

        What if these users already exist on the system?

        The package checks for existence of the user before it creates the users. If the users are already existed, the package will use the existing users.

        Show
        Eric Yang added a comment - 49 is not a safe gid on RHEL6. Base on Redhat 6 document, it should be available. What group uses 49? Reference: http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/s1-users-groups-standard-groups.html What if these users already exist on the system? The package checks for existence of the user before it creates the users. If the users are already existed, the package will use the existing users.
        Show
        Allen Wittenauer added a comment - What group uses 49? wnn uses it: http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/s1-users-groups-standard-users.html
        Hide
        Eric Yang added a comment -

        Thanks for the updated list. Checked the list again, and gid 114 seems to be available for Redhat and Debian.

        Show
        Eric Yang added a comment - Thanks for the updated list. Checked the list again, and gid 114 seems to be available for Redhat and Debian.
        Hide
        Eric Yang added a comment -

        Set HDFS to uid 201, mapred to uid 202, and hadoop group to gid 114 for branch-0.20-security.

        Show
        Eric Yang added a comment - Set HDFS to uid 201, mapred to uid 202, and hadoop group to gid 114 for branch-0.20-security.
        Hide
        Eric Yang added a comment -

        Same patch for trunk.

        Show
        Eric Yang added a comment - Same patch for trunk.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12496583/HADOOP-7603-trunk.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/233//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496583/HADOOP-7603-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/233//console This message is automatically generated.
        Hide
        Matt Foley added a comment -

        Thanks Eric. Committed to 0.20-sec and 0.20.205.

        Show
        Matt Foley added a comment - Thanks Eric. Committed to 0.20-sec and 0.20.205.
        Hide
        Eric Yang added a comment -

        I just committed this to 0.23 and trunk, thanks Allen and Matt.

        Show
        Eric Yang added a comment - I just committed this to 0.23 and trunk, thanks Allen and Matt.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1004 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1004/)
        HADOOP-7603. Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178644
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1004 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1004/ ) HADOOP-7603 . Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178644 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1082 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1082/)
        HADOOP-7603. Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178644
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1082 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1082/ ) HADOOP-7603 . Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178644 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1024 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1024/)
        HADOOP-7603. Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178644
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1024 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1024/ ) HADOOP-7603 . Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178644 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #36 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/36/)
        HADOOP-7603. Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178643
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #36 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/36/ ) HADOOP-7603 . Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178643 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #820 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/820/)
        HADOOP-7603. Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178644
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #820 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/820/ ) HADOOP-7603 . Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178644 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #29 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/29/)
        HADOOP-7603. Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178643
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #29 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/29/ ) HADOOP-7603 . Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178643 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #850 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/850/)
        HADOOP-7603. Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178644
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #850 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/850/ ) HADOOP-7603 . Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178644 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/hadoop.control/preinst /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/spec/hadoop-hdfs.spec /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/deb/hadoop.control/preinst /hadoop/common/trunk/hadoop-mapreduce-project/src/packages/rpm/spec/hadoop-mapred.spec
        Hide
        Matt Foley added a comment -

        Closed upon release of 0.20.205.0

        Show
        Matt Foley added a comment - Closed upon release of 0.20.205.0

          People

          • Assignee:
            Eric Yang
            Reporter:
            Eric Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development