[HDFS-12920] HDFS default value change (with adding time unit) breaks old version MR tarball work with Hadoop 3.x

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.4.0, 3.3.2
• Fix Version/s: 3.4.0, 3.2.3, 3.3.2
• Component/s: configuration, hdfs
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.2.3, 3.3.2
• Hadoop Flags: Reviewed
• Assignee: Akira Ajisaka
• Reporter: Junping Du
• Created: 13/Dec/17 00:58
• Updated: 27/Jan/24 03:31
• Resolved: 25/Jul/21 16:57

Description

After HADOOP-15059 get resolved. I tried to deploy 2.9.0 tar ball with 3.0.0 RC1, and run the job with following errors:

2017-12-12 13:29:06,824 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NumberFormatException: For input string: "30s"
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NumberFormatException: For input string: "30s"
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:542)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:522)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1764)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:522)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:308)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1722)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1719)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1650)

This is because HDFS-10845, we are adding time unit to hdfs-default.xml but it cannot be recognized by old version MR jars.
This break our rolling upgrade story, so should mark as blocker.
A quick workaround is to add values in hdfs-site.xml with removing all time unit. But the right way may be to revert HDFS-10845 (and get rid of noisy warnings).

Issue Links

• duplicates: HDFS-13889 The hadoop3.x client have compatible problem with hadoop2.x cluster
• is broken by: HDFS-10845 Change defaults in hdfs-site.xml to match timeunit type
• relates to: HADOOP-15059 3.0 deployment cannot work with old version MR tar ball which breaks rolling upgrade
• relates to: HDFS-14630 Configuration.getTimeDurationHelper() should not log time unit warning in info log.
• links to: GitHub Pull Request #3227