Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-7489

Hadoop logs errors upon startup on OS X 10.7

        Details

        • Type: Bug
        • Status: Resolved
        • Priority: Minor
        • Resolution: Won't Fix
        • Affects Version/s: None
        • Fix Version/s: None
        • Component/s: None
        • Labels:
          None
        • Environment:

          Mac OS X 10.7, Java 1.6.0_26

          Description

          When starting Hadoop on OS X 10.7 ("Lion") using start-all.sh, Hadoop logs the following errors:
          2011-07-28 11:45:31.469 java[77427:1a03] Unable to load realm info from SCDynamicStore
          Hadoop does seem to function properly despite this.

            Issue Links

            is cloned by

            Improvement - An improvement or enhancement to an existing feature or task. HADOOP-8719 Workaround for kerberos-related log errors upon running any hadoop command on OSX

            • Trivial - Cosmetic problem like misspelt words or misaligned text.
            • Resolved
            is related to

            Test - A new unit, integration or system test. PIG-2740 get rid of "java[77427:1a03] Unable to load realm info from SCDynamicStore" log lines when running pig tests

            • Major - Major loss of function.
            • Closed

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              buckett Matthew Buckett added a comment -

              I'm also seeing this on another project (on OS X 10.7) and it's happening when I attempt to perform a login using a keytab file. No idea why yet.

              Show
              buckett Matthew Buckett added a comment - I'm also seeing this on another project (on OS X 10.7) and it's happening when I attempt to perform a login using a keytab file. No idea why yet.
              Hide
              Permalink
              pkilovatiy Pavel Kilovatiy added a comment -

              Having exactly the same problem on the OS X 10.7.1/JDK 1.6.0_26, Hadoop 0.20.204.0, pseudo-distributed mode.

              NameNode daemon seems fail to start - run "jps" to verify. Also "stop_all.sh" outputs "no namenode to stop"

              Show
              pkilovatiy Pavel Kilovatiy added a comment - Having exactly the same problem on the OS X 10.7.1/JDK 1.6.0_26, Hadoop 0.20.204.0, pseudo-distributed mode. NameNode daemon seems fail to start - run "jps" to verify. Also "stop_all.sh" outputs "no namenode to stop"
              Hide
              Permalink
              pkilovatiy Pavel Kilovatiy added a comment -

              Please disregard previous comment about NameNode daemon not starting - it was a separate issue not related to this specific problem.

              Show
              pkilovatiy Pavel Kilovatiy added a comment - Please disregard previous comment about NameNode daemon not starting - it was a separate issue not related to this specific problem.
              Hide
              Permalink
              buckett Matthew Buckett added a comment -

              I think the OS X JDK is trying to get some information from configd which is failing that causes the whole LoginContent to break.

              This can be worked around by specifying kerberos config on the command line:

              -Djava.security.krb5.realm=OX.AC.UK -Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk

              Show
              buckett Matthew Buckett added a comment - I think the OS X JDK is trying to get some information from configd which is failing that causes the whole LoginContent to break. This can be worked around by specifying kerberos config on the command line: -Djava.security.krb5.realm=OX.AC.UK -Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk
              Hide
              Permalink
              bmargulies Benson Margulies added a comment -

              What realm do we use for a standalone machine?

              Show
              bmargulies Benson Margulies added a comment - What realm do we use for a standalone machine?
              Hide
              Permalink
              steven.borrelli Steven Borrelli added a comment -

              For a standalone machine, you can just set the values to nothing.

              Adding the following to your environment or hadoop-env.sh should suppress the errors:

              export HADOOP_OPTS="-Djava.security.krb5.realm= -Djava.security.krb5.kdc="

              Show
              steven.borrelli Steven Borrelli added a comment - For a standalone machine, you can just set the values to nothing. Adding the following to your environment or hadoop-env.sh should suppress the errors: export HADOOP_OPTS="-Djava.security.krb5.realm= -Djava.security.krb5.kdc="
              Hide
              Permalink
              aw Allen Wittenauer added a comment -

              Java has its own internal version of Kerberos. That version is very, very stupid in 1.6 and earlier when it comes to using naming services for auto-discovery of the realm and KDC information. You'll see similar weirdness even on non-OS X boxes when the krb5.conf doesn't explicitly list the realm information. The same configuration fix mentioned here applies there as well. This has been fixed in JRE 1.7. Allegedly.

              Show
              aw Allen Wittenauer added a comment - Java has its own internal version of Kerberos. That version is very, very stupid in 1.6 and earlier when it comes to using naming services for auto-discovery of the realm and KDC information. You'll see similar weirdness even on non-OS X boxes when the krb5.conf doesn't explicitly list the realm information. The same configuration fix mentioned here applies there as well. This has been fixed in JRE 1.7. Allegedly.
              Hide
              Permalink
              aw Allen Wittenauer added a comment -

              I'm closing this as Won't Fix since this is a known JRE bug and not particularly anything we can do about it in Hadoop.

              Show
              aw Allen Wittenauer added a comment - I'm closing this as Won't Fix since this is a known JRE bug and not particularly anything we can do about it in Hadoop.
              Hide
              Permalink
              vishnoiprem prem added a comment -

              Same error i am geting on mac 10.11.2 with java 1.8 and hadopp 2.7

              Show
              vishnoiprem prem added a comment - Same error i am geting on mac 10.11.2 with java 1.8 and hadopp 2.7

                People

                • Assignee:
                  Unassigned
                  Reporter:
                  bryanck Bryan Keller
                • Votes:
                  4 Vote for this issue
                  Watchers:
                  15 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development