Hadoop Common
  1. Hadoop Common
  2. HADOOP-7489

Hadoop logs errors upon startup on OS X 10.7

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Mac OS X 10.7, Java 1.6.0_26

      Description

      When starting Hadoop on OS X 10.7 ("Lion") using start-all.sh, Hadoop logs the following errors:
      2011-07-28 11:45:31.469 java[77427:1a03] Unable to load realm info from SCDynamicStore
      Hadoop does seem to function properly despite this.

        Issue Links

          Activity

          Bryan Keller created issue -
          Hide
          Matthew Buckett added a comment -

          I'm also seeing this on another project (on OS X 10.7) and it's happening when I attempt to perform a login using a keytab file. No idea why yet.

          Show
          Matthew Buckett added a comment - I'm also seeing this on another project (on OS X 10.7) and it's happening when I attempt to perform a login using a keytab file. No idea why yet.
          Hide
          Pavel Kilovatiy added a comment -

          Having exactly the same problem on the OS X 10.7.1/JDK 1.6.0_26, Hadoop 0.20.204.0, pseudo-distributed mode.

          NameNode daemon seems fail to start - run "jps" to verify. Also "stop_all.sh" outputs "no namenode to stop"

          Show
          Pavel Kilovatiy added a comment - Having exactly the same problem on the OS X 10.7.1/JDK 1.6.0_26, Hadoop 0.20.204.0, pseudo-distributed mode. NameNode daemon seems fail to start - run "jps" to verify. Also "stop_all.sh" outputs "no namenode to stop"
          Hide
          Pavel Kilovatiy added a comment -

          Please disregard previous comment about NameNode daemon not starting - it was a separate issue not related to this specific problem.

          Show
          Pavel Kilovatiy added a comment - Please disregard previous comment about NameNode daemon not starting - it was a separate issue not related to this specific problem.
          Hide
          Matthew Buckett added a comment -

          I think the OS X JDK is trying to get some information from configd which is failing that causes the whole LoginContent to break.

          This can be worked around by specifying kerberos config on the command line:

          -Djava.security.krb5.realm=OX.AC.UK -Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk

          Show
          Matthew Buckett added a comment - I think the OS X JDK is trying to get some information from configd which is failing that causes the whole LoginContent to break. This can be worked around by specifying kerberos config on the command line: -Djava.security.krb5.realm=OX.AC.UK -Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk
          Hide
          Benson Margulies added a comment -

          What realm do we use for a standalone machine?

          Show
          Benson Margulies added a comment - What realm do we use for a standalone machine?
          Hide
          Steven Borrelli added a comment -

          For a standalone machine, you can just set the values to nothing.

          Adding the following to your environment or hadoop-env.sh should suppress the errors:

          export HADOOP_OPTS="-Djava.security.krb5.realm= -Djava.security.krb5.kdc="

          Show
          Steven Borrelli added a comment - For a standalone machine, you can just set the values to nothing. Adding the following to your environment or hadoop-env.sh should suppress the errors: export HADOOP_OPTS="-Djava.security.krb5.realm= -Djava.security.krb5.kdc="
          Hide
          Allen Wittenauer added a comment -

          Java has its own internal version of Kerberos. That version is very, very stupid in 1.6 and earlier when it comes to using naming services for auto-discovery of the realm and KDC information. You'll see similar weirdness even on non-OS X boxes when the krb5.conf doesn't explicitly list the realm information. The same configuration fix mentioned here applies there as well. This has been fixed in JRE 1.7. Allegedly.

          Show
          Allen Wittenauer added a comment - Java has its own internal version of Kerberos. That version is very, very stupid in 1.6 and earlier when it comes to using naming services for auto-discovery of the realm and KDC information. You'll see similar weirdness even on non-OS X boxes when the krb5.conf doesn't explicitly list the realm information. The same configuration fix mentioned here applies there as well. This has been fixed in JRE 1.7. Allegedly.
          Hide
          Allen Wittenauer added a comment -

          I'm closing this as Won't Fix since this is a known JRE bug and not particularly anything we can do about it in Hadoop.

          Show
          Allen Wittenauer added a comment - I'm closing this as Won't Fix since this is a known JRE bug and not particularly anything we can do about it in Hadoop.
          Allen Wittenauer made changes -
          Field Original Value New Value
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Won't Fix [ 2 ]
          Julien Le Dem made changes -
          Link This issue is related to PIG-2740 [ PIG-2740 ]
          Jianbin Wei made changes -
          Link This issue is cloned as HADOOP-8719 [ HADOOP-8719 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              Bryan Keller
            • Votes:
              4 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development