[AMBARI-7844] Ambari needs to fix client configuration files to point to versioned tarballs

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Invalid
• Affects Version/s: 1.7.0
• Fix Version/s: 1.7.0
• Component/s: ambari-server
• Assignee: Alejandro Fernandez
• Reporter: Alejandro Fernandez
• Created: 17/Oct/14 22:13
• Updated: 24/Oct/14 18:55
• Resolved: 24/Oct/14 18:55

Description

Once AMBARI-7842 is committed, many clients in the stack need to point to the versioned tar-balls on HDFS. Ambari needs to propagate these changes to the clients by modifying the client configs to point to the right tar-ball version.

This will require dynamic variable interpretation in the *-site.xml files

Client where the config should be changed	Configuration file	Configuration property	Example value
MR clients	mapred-site.xml	mapreduce.application.framework.path	hdfs:/mapred/framework/hadoop-2.3.0.tar.gz#mr-framework
MR clients	mapred-site.xml	mapreduce.application.classpath	$PWD/mr-framework/hadoop-2.3.0/share/hadoop/mapreduce/:$PWD/mr-framework/hadoop-2.3.0/share/hadoop/mapreduce/lib/:$PWD/mr-framework/hadoop-2.3.0/share/hadoop/common/:$PWD/mr-framework/hadoop-2.3.0/share/hadoop/common/lib/:$PWD/mr-framework/hadoop-2.3.0/share/hadoop/yarn/:$PWD/mr-framework/hadoop-2.3.0/share/hadoop/yarn/lib/:$PWD/mr-framework/hadoop-2.3.0/share/hadoop/hdfs/:$PWD/mr-framework/hadoop-2.3.0/share/hadoop/hdfs/lib/
Tez client	tez-site.xml	tez.lib.uris	(fs.defaultFS)/hdp/hdp_version/apps/tez/tez.tar.gz
Tez client	tez-site.xml	tez.aux.uris	(fs.defaultFS)/hdp/hdp_version/apps/tez/aux/hadoop-lzo.jar
WebHCat server	webhcat-site.xml	templeton.pig.archive	hdfs:///apps/templeton/pig-$(env.PIG_VERSION).tar.gz
WebHCat server	webhcat-site.xml	templeton.pig.path	pig-$(env.PIG_VERSION).tar.gz/pig-$(env.PIG_VERSION)/bin/pig
WebHCat server	webhcat-site.xml	templeton.hive.archive	hdfs:///apps/templeton/apache-hive-$(env.HIVE_VERSION)-bin.tar.gz
WebHCat server	webhcat-site.xml	templeton.hive.path	apache-hive-$(env.HIVE_VERSION)bin.tar.gz/apache-hive$(env.HIVE_VERSION)-bin/bin/hive
WebHCat server	webhcat-site.xml	templeton.hive.home	apache-hive-$(env.HIVE_VERSION)bin.tar.gz/apache-hive$(env.HIVE_VERSION)-bin
WebHCat server	webhcat-site.xml	templeton.hcat.home	apache-hive-$(env.HIVE_VERSION)bin.tar.gz/apache-hive$(env.HIVE_VERSION)-bin/hcatalog
WebHCat server	webhcat-site.xml	templeton.sqoop.home	-
WebHCat server	webhcat-site.xml	templeton.sqoop.path	-
WebHCat server	webhcat-site.xml	templeton.sqoop.archive	-
Slider	N/A	N/A	Slider client uploads all the dependencies for each application instance

Oozie server has its own shared lib mechanism that is already integrated into Ambari, is versioned and is not easy to integrate into a unified layout.

WebHCat related tarballs are already uploaded to HDFS by the installer.

Issue Links

• is blocked by: AMBARI-7842 Ambari to manage tarballs on HDFS