[AMBARI-20685] Upgrade Progress Dialog Executes Query Which Causes StackOverflow in JPA

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.4.2
• Fix Version/s: 2.5.1
• Component/s: ambari-server
• Assignee: Jonathan Hurley
• Reporter: Jonathan Hurley
• Created: 05/Apr/17 20:28
• Updated: 16/Feb/18 13:14
• Resolved: 06/Apr/17 15:48

Description

Large rolling upgrades (1000 hosts with 10,000 host components) creates a massive request object object with 10's of 1000's of stages and tasks. The web client executes a GET command against the API to retrieve upgrade groups/items. On a large upgrade, this causes the EclipseLink SQL generation code to run out of stack frame space while recursively building a query. To work around this, the stack frame size per thread can be increased using -Xss along with a value like 32M.

http://localhost:8080/api/v1/clusters/c1/upgrades/12?
upgrade_groups/UpgradeGroup/status!=PENDING&
fields=
  Upgrade/progress_percent,
  Upgrade/request_context,
  Upgrade/request_status,
  Upgrade/direction,
  Upgrade/downgrade_allowed,
  upgrade_groups/UpgradeGroup,
  Upgrade/*,
  upgrade_groups/upgrade_items/UpgradeItem/status,
  upgrade_groups/upgrade_items/UpgradeItem/display_status,
  upgrade_groups/upgrade_items/UpgradeItem/context,
  upgrade_groups/upgrade_items/UpgradeItem/group_id,
  upgrade_groups/upgrade_items/UpgradeItem/progress_percent,
  upgrade_groups/upgrade_items/UpgradeItem/request_id,
  upgrade_groups/upgrade_items/UpgradeItem/skippable,
  upgrade_groups/upgrade_items/UpgradeItem/stage_id,
  upgrade_groups/upgrade_items/UpgradeItem/text&
  minimal_response=true&_=1489680258782

This call gets turn into a LOT of queries, but some seem to be based on the number of stages. For example, this one on a very small upgrade produces very poor SQL...

SELECT
  stage_id,
  cluster_host_info,
  cluster_id,
  command_execution_type,
  command_params,
  display_status,
  host_params,
  log_info,
  request_context,
  request_id,
  skippable,
  status,
  supports_auto_skip_failure
FROM stage
WHERE ((request_id = ?)
AND ((((((((stage_id = ?)
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?))
OR (stage_id = ?)))

Issue Links

• links to: Reviewboard