[HDFS-9471] Webhdfs not working with shell command when kerberos security+https is enabled.

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Not A Problem
• Affects Version/s: 2.7.1
• Component/s: webhdfs
• Assignee: Surendra Singh Lilhore
• Reporter: Surendra Singh Lilhore
• Created: 26/Nov/15 14:50
• Updated: 30/Nov/15 17:18
• Resolved: 30/Nov/15 17:18

Description

Client exception

secure@host85:/opt/hdfsdata/HA/install/hadoop/namenode/bin> ./hdfs dfs -ls webhdfs://x.x.x.x:50070/test
15/11/25 18:46:55 ERROR web.WebHdfsFileSystem: Unable to get HomeDirectory from original File System
java.net.SocketException: Unexpected end of file from server
        at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:792)

Exception in namenode log

2015-11-26 11:03:18,231 WARN org.mortbay.log: EXCEPTION
javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
        at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:710)
        at sun.security.ssl.InputRecord.read(InputRecord.java:527)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:961)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1363)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1391)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1375)
        at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:708)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

This is because URL schema hard coded in WebHdfsFileSystem.getTransportScheme().

 /**
   * return the underlying transport protocol (http / https).
   */
  protected String getTransportScheme() {
    return "http";
  }

Issue Links

• relates to: HDFS-9483 Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS.