[HDFS-5295] hsftp throws an exception in the end on secure cluster with https enabled

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.1.1-beta, 3.0.0-alpha1

Environment

Secure

• Assignee: Haohui Mai
• Reporter: Yesha Vora
• Created: 03/Oct/13 19:52
• Updated: 12/May/16 18:14
• Resolved: 12/Nov/13 02:55

Description

The hsftp command throws "java.net.SocketException: Unexpected end of file from server" exception in the end in secure environment and with https enabled.

Using https port defined by dfs.https.port=50701

/usr/bin/hdfs dfs -fs hdfs://hostname:8020 -ls -R "hsftp://hostname:50701/user/abc/1380829410/.abc.crc"
rw-rr- 3 abc abc 12 2013-10-03 19:44 hsftp://hostname:50701/user/abc/1380829410/.abc.crc
13/10/03 19:50:48 INFO tools.DelegationTokenFetcher: error in cancel over HTTP
java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:718)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:579)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:715)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:579)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1322)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.isNegotiate(KerberosAuthenticator.java:224)
at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:194)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:232)
at org.apache.hadoop.security.SecurityUtil.openSecureHttpConnection(SecurityUtil.java:512)
at org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.cancelDelegationToken(DelegationTokenFetcher.java:354)
at org.apache.hadoop.hdfs.HftpFileSystem$TokenManager.cancel(HftpFileSystem.java:730)
at org.apache.hadoop.security.token.Token.cancel(Token.java:384)
at org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.cancel(DelegationTokenRenewer.java:152)
at org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.access$200(DelegationTokenRenewer.java:58)
at org.apache.hadoop.fs.DelegationTokenRenewer.removeRenewAction(DelegationTokenRenewer.java:241)
at org.apache.hadoop.hdfs.HftpFileSystem.close(HftpFileSystem.java:417)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2524)
at org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2541)
at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
13/10/03 19:50:48 INFO fs.FileSystem: FileSystem.Cache.closeAll() threw an exception:
java.net.SocketException: Unexpected end of file from server

Issue Links

• is duplicated by: HDFS-5502 Fix HTTPS support in HsftpFileSystem