[AMBARI-14751] Ambari not setting hdp.version in config files

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Cannot Reproduce
• Affects Version/s: 2.2.0
• Fix Version/s: 2.2.2
• Assignee: Dmitry Lysnichenko
• Reporter: Nick Dimiduk
• Created: 21/Jan/16 00:05
• Updated: 22/Mar/16 01:14
• Resolved: 22/Mar/16 01:14

Description

After dealing with a number of arcane failures, I've tracked down the issue to ambari not rendering config files correctly. This manifests mostly MR and Spark job submission failures. For example,

$ hbase org.apache.hadoop.hbase.mapreduce.RowCounter -libjars /usr/hdp/current/phoenix-client/phoenix-server.jar -Dhbase.client.scanner.caching=1000 -Dmapreduce.map.speculative=false TEST
...
Exception in thread "main" java.lang.IllegalArgumentException: Unable to parse '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a URI, check the setting for mapreduce.application.framework.path
        at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:443)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:142)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
        at org.apache.hadoop.hbase.mapreduce.RowCounter.main(RowCounter.java:210)
Caused by: java.net.URISyntaxException: Illegal character in path at index 11: /hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework
        at java.net.URI$Parser.fail(URI.java:2848)
        at java.net.URI$Parser.checkChars(URI.java:3021)
        at java.net.URI$Parser.parseHierarchical(URI.java:3105)
        at java.net.URI$Parser.parse(URI.java:3063)
        at java.net.URI.<init>(URI.java:588)
        at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:441)
        ... 9 more

Restarting ambari-server + redeploying configs does not help.

$ dpkg -s ambari-server
Package: ambari-server
Status: install ok installed
Priority: extra
Section: universe/admin
Installed-Size: 457149
Maintainer: Hortonworks
Architecture: amd64
Version: 2.2.0.0-1310
Depends: openssl, postgresql (>= 8.1), python (>= 2.6), curl
Description: Ambari Server