Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1828

Introduce counters JobStatus terminal states metrics

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.0.0
    • Fix Version/s: 4.1.0
    • Component/s: monitoring
    • Labels:
      None

      Description

      Currently the Oozie server exposes job status metrics from the 'variables' group. These include metrics for jobs in terminal states: 'SUCCEEDED', 'FAILED', 'KILLED'. The way Oozie compute the metrics is by querying the database for all jobs in each and every state. This means that when a purge happens these "apparent" counters' values are going to change (if anything was purged). This renders these counters as not very useful.

      It would be better if real counters for jobs in terminal states can be exposed from the oozie server. One way to do this would be to initialize an in-memory counters and a timestamp count the jobs that finished between the timestamp and 'now' (and keep updating timestamp to avoid it falling out of the retention period). This means that each Oozie server may have its own counter but that is okay as the count itself is not very important what is important is the rate-of-change.

        Attachments

        1. OOZIE-1828.patch
          1 kB
          Robert Kanter
        2. OOZIE-1828.patch
          3 kB
          Robert Kanter
        3. OOZIE-1828.patch
          3 kB
          Robert Kanter

          Activity

            People

            • Assignee:
              rkanter Robert Kanter
              Reporter:
              giladwolff Gilad Wolff
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: