[AMBARI-16835] WEBHDFS PUT fails on Amazon Linux

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.4.0
• Assignee: Andrew Onischuk
• Reporter: Andrew Onischuk
• Created: 24/May/16 09:25
• Updated: 26/May/16 19:45
• Resolved: 26/May/16 16:53

Description

If kerberos is enabled on the cluster the WEBHDFS PUT requests are failing
thus the Historyserver fails to start and copy the mapreduce.tar.gz to HDFS.
This is caused by the curl patched by Amazon.

OS: Amazon Linux AMI release 2016.03

uname: Linux ip-10-0-216-251 4.4.8-20.46.amzn1.x86_64 #1 SMP Wed Apr 27 19:28:52 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

curl --version: curl 7.40.0 (x86_64-redhat-linux-gnu) libcurl/7.40.0 NSS/3.19.1 Basic ECC zlib/1.2.8 libidn/1.18 libssh2/1.4.2
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz UnixSockets

The command that is issued by the historyserver -with verbose switch:
curl1.log
The same command with the proper curl version on the same host: curl2.log

This seems to be caused by a bug in curl on amazon2016 making curl hang when
uploading files with curl larger than 1MB.

Issue Links

• links to: ReviewBoard