[HADOOP-12671] Inconsistent s3a configuration values and incorrect comments

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.7.1, 2.6.2
• Component/s: conf, documentation, fs/s3
• Assignee: Tianyin Xu
• Reporter: Tianyin Xu
• Created: 22/Dec/15 23:14
• Updated: 08/Aug/16 06:18

Description

The two values in core-default.xml are wrong.
fs.s3a.multipart.purge.age
fs.s3a.connection.timeout
fs.s3a.connection.establish.timeout

1. fs.s3a.multipart.purge.age
(in both 2.6.2 and 2.7.1)
In core-default.xml , the value is 86400 (24 hours), while in the code it is 14400 (4 hours).

2. fs.s3a.connection.timeout
(only appear in 2.6.2)
In core-default.xml (2.6.2) , the value is 5000, while in the code it is 50000.

  // seconds until we give up on a connection to s3
  public static final String SOCKET_TIMEOUT = "fs.s3a.connection.timeout";
  public static final int DEFAULT_SOCKET_TIMEOUT = 50000;

3. fs.s3a.connection.establish.timeout
(only appear in 2.7.1)
In core-default.xml (2.7.1), the value is 5000, while in the code it is 50000.

  // seconds until we give up trying to establish a connection to s3
  public static final String ESTABLISH_TIMEOUT = "fs.s3a.connection.establish.timeout";
  public static final int DEFAULT_ESTABLISH_TIMEOUT = 50000;

btw, the code comments are wrong! The two parameters are in the unit of milliseconds instead of seconds...

-  // seconds until we give up on a connection to s3
+  // milliseconds until we give up on a connection to s3
...
-  // seconds until we give up trying to establish a connection to s3
+  // milliseconds until we give up trying to establish a connection to s3

Issue Links

• is duplicated by: HADOOP-13472 Inconsistencies for s3a timeouts in description and default values
• is related to: HADOOP-11521 Make connection timeout configurable in s3a
• is related to: HADOOP-12982 Document missing S3A and S3 properties