Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3077 Quorum-based protocol for reading and writing edit logs
  3. HDFS-3904

QJM: journalnode does not die/log ERROR when keytab is not found in secure mode

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: QuorumJournalManager (HDFS-3077)
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Credit to Stephen Chu for finding this. The journalnode was incorrectly configured (misplaced the keytab) with security enabled, when started the JournalNode didn't die. It stayed running and logged a WARN message:

      2012-08-23 15:44:15,497 WARN org.mortbay.log: Failed startup of context org.mortbay.jetty.webapp.WebAppContext@58c16b18{/,file:/usr/lib/hadoop-hdfs/webapps/journal}
      javax.servlet.ServletException: javax.servlet.ServletException: Keytab does not exist: /etc/hadoop/conf/hdfs.keytab
        at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:185)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:146)
        at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)
        at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
        at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
        at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
        at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
        at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
        at org.mortbay.jetty.Server.doStart(Server.java:224)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.apache.hadoop.http.HttpServer.start(HttpServer.java:657)
        at org.apache.hadoop.hdfs.qjournal.server.JournalNodeHttpServer.start(JournalNodeHttpServer.java:83)
        at org.apache.hadoop.hdfs.qjournal.server.JournalNode.start(JournalNode.java:138)
        at org.apache.hadoop.hdfs.qjournal.server.JournalNode.run(JournalNode.java:120)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.hadoop.hdfs.qjournal.server.JournalNode.main(JournalNode.java:228)
      Caused by: javax.servlet.ServletException: Keytab does not exist: /etc/hadoop/conf/hdfs.keytab
        at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:153)
        ... 22 more
      

      The other HDFS daemons, if I remember correctly, would die if they can't authenticate.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                eli Eli Collins
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: