[AMBARI-15686] Alert Dispatch Scheduling Changes to Support Repeat Tolerance

• Type: Task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.4.0
• Component/s: ambari-server
• Assignee: Jonathan Hurley
• Reporter: Jonathan Hurley
• Created: 04/Apr/16 15:32
• Updated: 04/Apr/16 23:50
• Resolved: 04/Apr/16 19:50

Description

When an alert state change is received, its notification will no longer be queued for immediate delivery if the firmness of the alert is SOFT. Instead, notifications will only be created and queued for delivery if the alert firmness is HARD in order to prevent false positives on alerts which have repeat tolerance enabled.

Changes Included in this task:

• Database changes to support HARD/SOFT states to complement repeat occurrence values

  CREATE TABLE alert_current (
    ...
    occurrences BIGINT NOT NULL DEFAULT 1,
    firmness VARCHAR(255) NOT NULL DEFAULT 'HARD',
    ...
  );
  

• Changes to eventing system to handle alert state changes difference depending on whether the alerts are HARD/SOFT
• HARD/SOFT Calculations
  • If an alert is AlertState#OK, then the firmness is always AlertFirmness#HARD
  • If an alert is link SourceType#AGGREGATE, then the firmness is always link AlertFirmness#HARD.
  • Otherwise, the firmness will be {@link AlertFirmness#SOFT}

    unless the repeat tolerance has been met.

• Exposure of HARD/SOFT alert states

  GET api/v1/clusters/c1/alerts/1
  {
    "href": "http://localhost:8080/api/v1/clusters/c1/alerts/1",
    "Alert": {
      "cluster_name": "c1",
      ...
      "repeat_tolerance": 1,
      "repeat_tolerance_remaining": 0,
      "occurrences": 8,
      "firmness": "HARD"
      ....
  

Issue Links

• links to: Reviewboard