[HDFS-9887] WebHdfs socket timeouts should be configurable

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: fs, webhdfs
• Labels: easyfix newbie

Environment

all

• Target Version/s: 2.8.0
• Hadoop Flags: Reviewed
• Release Note: Added new configuration options: dfs.webhdfs.socket.connect-timeout and dfs.webhdfs.socket.read-timeout both defaulting to 60s.
• Flags: Patch
• Assignee: Austin Donnelly
• Reporter: Austin Donnelly
• Created: 19/Feb/16 15:47
• Updated: 02/Oct/19 17:13
• Resolved: 02/Mar/16 17:27

Description

WebHdfs client connections use sockets with fixed timeouts of 60 seconds to connect, and 60 seconds for reads.
This is a problem because I am trying to use WebHdfs to access an archive storage system which can take minutes to hours to return the requested data over WebHdfs.

The fix is to add new configuration file options to allow these 60s defaults to be customised in hdfs-site.xml.

If the new configuration options are not present, the behavior is unchanged from before.

Issue Links

• breaks: HDFS-9914 Fix configurable WebhDFS connect/read timeout