Hadoop Common
  1. Hadoop Common
  2. HADOOP-7750

DataNode: Cannot start secure cluster without privileged resources | tags/release-0.20.205.0-rc2

    Details

    • Tags:
      release-0.20.205.0-rc2

      Description

      This tag compiles just fine. But after configuring it, the datanode fails on startup with the below error:

      TARTUP_MSG: Starting DataNode
      STARTUP_MSG: host = hd3w94m7/10.152.94.111
      STARTUP_MSG: args = []
      STARTUP_MSG: version = 0.20.205.1
      STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.205.0-rc2 -r 1179942; compiled by 'tpowell1' on Wed Oct 12 11:14:46 PDT 2011
      ************************************************************/
      2011-10-14 15:24:56,028 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
      2011-10-14 15:24:56,043 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
      2011-10-14 15:24:56,044 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
      2011-10-14 15:24:56,044 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started
      2011-10-14 15:24:56,192 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered.
      2011-10-14 15:24:56,421 INFO org.apache.hadoop.security.UserGroupInformation: Asked the TGT renewer thread to terminate
      2011-10-14 15:24:57,241 INFO org.apache.hadoop.security.UserGroupInformation: Login successful for user hdfs/hd3w94m7@XXX using keytab file /home/tpowell1/hadoop.tags.release-0.20.205.0-rc2/conf/hdfs.keytab
      2011-10-14 15:24:57,242 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.lang.RuntimeException: Cannot start secure cluster without privileged resources.
      at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:306)
      at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:281)
      at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1545)
      at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1484)
      at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1502)
      at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1628)
      at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1645)

      2011-10-14 15:24:57,243 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
      /************************************************************
      SHUTDOWN_MSG: Shutting down DataNode at hd3w94m7.XXX/10.152.94.111
      ************************************************************/

      Checking the Datanode.java code it is started with a null SecureResources .

      public static void main(String args[])

      { secureMain(args, null); }

      This null resource seems to get passed all the way down to startDataNode() where there is a null check... which in turns throws the error we see.

      void startDataNode(Configuration conf,
      AbstractList<File> dataDirs, SecureResources resources
      ) throws IOException {
      if(UserGroupInformation.isSecurityEnabled() && resources == null)
      throw new RuntimeException("Cannot start secure cluster without " +
      "privileged resources.");

        Activity

        Hide
        Aaron T. Myers added a comment -

        Hi Trevor, is jsvc definitely accessible/executable in the package you built? Did you definitely have some low ports configured for the DN?

        Show
        Aaron T. Myers added a comment - Hi Trevor, is jsvc definitely accessible/executable in the package you built? Did you definitely have some low ports configured for the DN?
        Hide
        Trevor Powell added a comment -

        Good Day,
        I have jsvc under $hadoophome/libexec/

        $ ls -l
        total 100
        -rwxr-xr-x 1 root root 52776 Oct 13 10:51 jsvc.amd64
        -rwxr-xr-x 1 root root 42624 Oct 13 10:51 jsvc.i386

        I figured this was good looking at the hadoop script.
        hadoop: exec "$HADOOP_HOME/libexec/jsvc.$

        {JSVC_ARCH}

        "

        As for my DN ports I have this:

        <!-- DataNode security config -->
        <property>
        <name>dfs.datanode.address</name>
        <value>0.0.0.0:1004</value>
        </property>
        <property>
        <name>dfs.datanode.http.address</name>
        <value>0.0.0.0:1006</value>
        </property>

        Thoughts?

        Show
        Trevor Powell added a comment - Good Day, I have jsvc under $hadoophome/libexec/ $ ls -l total 100 -rwxr-xr-x 1 root root 52776 Oct 13 10:51 jsvc.amd64 -rwxr-xr-x 1 root root 42624 Oct 13 10:51 jsvc.i386 I figured this was good looking at the hadoop script. hadoop: exec "$HADOOP_HOME/libexec/jsvc.$ {JSVC_ARCH} " As for my DN ports I have this: <!-- DataNode security config --> <property> <name>dfs.datanode.address</name> <value>0.0.0.0:1004</value> </property> <property> <name>dfs.datanode.http.address</name> <value>0.0.0.0:1006</value> </property> Thoughts?
        Hide
        Aaron T. Myers added a comment -

        Did you start the DN as root? The DN won't be able to bind to low ports (< 1024) without being started as root.

        Show
        Aaron T. Myers added a comment - Did you start the DN as root? The DN won't be able to bind to low ports (< 1024) without being started as root.
        Hide
        Trevor Powell added a comment -

        Yes.
        I run "sudo hadoop-daemon.sh start datanode"

        Show
        Trevor Powell added a comment - Yes. I run "sudo hadoop-daemon.sh start datanode"
        Hide
        Trevor Powell added a comment -

        I see "tags/release-0.20.205.0-rc2" is not in SVN anymore and there is a release of "0.20.205.0" on the general site.
        I will download this and try it out.

        Show
        Trevor Powell added a comment - I see "tags/release-0.20.205.0-rc2" is not in SVN anymore and there is a release of "0.20.205.0" on the general site. I will download this and try it out.
        Hide
        fujie added a comment -

        i found this problem in 1.0.3-release

        Show
        fujie added a comment - i found this problem in 1.0.3-release

          People

          • Assignee:
            Unassigned
            Reporter:
            Trevor Powell
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development