[AMBARI-16651] Ambari 2.2.0 breaks backward compatibility with HDP 2.3.2 and below due to spark configs with hdp.version being removed

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.4.0
• Component/s: stacks
• Assignee: Jayush Luniya
• Reporter: Jayush Luniya
• Created: 13/May/16 06:05
• Updated: 15/Jun/16 22:07
• Resolved: 15/Jun/16 19:54

Description

In Ambari 2.2.0 few Spark properties were removed. This is because HDP 2.3.4 (Spark 1.5+) self-detects hdp version.

spark.driver.extraJavaOptions -Dhdp.version=
spark.yarn.am.extraJavaOptions -Dhdp.version=
However, removing these properties causes Ambari 2.2.0 + HDP 2.3.2 (Spark 1.4) to break as those properties were required.

This JIRA is to address this backward compatibility issue with HDP 2.3.2 and below caused after upgrading to Ambari 2.2.0

Issue Links

• is duplicated by: AMBARI-17244 Ambari 2.2.0 breaks backward compatibility with HDP 2.3.2 and below due to spark configs with hdp.version being removed
• links to: RB