Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-733

Add metrics for Elasticsearch Producer

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: metrics
    • Labels:
      None

      Description

      Add metrics for Elasticsearch producers. For now, we'll only track success events (number of successful bulk sends, number of documents inserted, and number of documents updated). Since the current ES producer fails the job when ES returns an error, there's no benefit yet to tracking negative events because Samza does not seem to flush metrics before shutting down (at least when I tested with ThreadJob).

      I've implemented metrics here (https://github.com/Quantiply/rico/tree/master/samza-elasticsearch/src/main/java/org/apache/samza/system/elasticsearch) and will work on creating a patch.

        Attachments

        1. SAMZA-733-WITH-METRICS-BASE-CLASS.patch
          17 kB
          Roger Hoover
        2. SAMZA-733-WITH-METRICS-BASE-CLASS.patch
          17 kB
          Roger Hoover
        3. SAMZA-733-WITH-METRICS-BASE-CLASS.patch
          17 kB
          Roger Hoover
        4. SAMZA-733.patch
          6 kB
          Roger Hoover
        5. SAMZA-733.patch
          6 kB
          Roger Hoover
        6. SAMZA-733.patch
          8 kB
          Roger Hoover

          Activity

            People

            • Assignee:
              theduderog Roger Hoover
              Reporter:
              theduderog Roger Hoover
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: