[AMBARI-20490] Fixes for Express Upgrade on large-scale clusters: batch execute-stage for hdp-select set all, EU from HDP 2.2 to 2.5

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Do
• Affects Version/s: 2.4.0
• Fix Version/s: trunk
• Component/s: ambari-server
• Assignee: Alejandro Fernandez
• Reporter: Alejandro Fernandez
• Created: 17/Mar/17 23:12
• Updated: 06/Apr/17 00:02
• Resolved: 06/Apr/17 00:02

Description

On large-scale clusters over 1000 nodes, performance during Express Upgrade can be very slow and it takes more time to fix potential errors.
We can remedy these with the following fixes:

• "hdp-select set all" is currently ran in the ClusterGroup with a single stage instead of batching. This can cause Ambari Server to be the bottleneck when processing 1000+ requests. Allow the execute-stage inside a ClusterGroup to use batching.

• There is currently no upgrade path from HDP 2.2 directly to 2.5. Doing a 2-step upgrade is very time consuming (only for branch 2.4 since HDP 2.2 no longer supported in Ambari 2.5).

Issue Links

• links to: Review Board patch