Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-733

Add metrics for Elasticsearch Producer

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 0.10.0
    • metrics
    • None

    Description

      Add metrics for Elasticsearch producers. For now, we'll only track success events (number of successful bulk sends, number of documents inserted, and number of documents updated). Since the current ES producer fails the job when ES returns an error, there's no benefit yet to tracking negative events because Samza does not seem to flush metrics before shutting down (at least when I tested with ThreadJob).

      I've implemented metrics here (https://github.com/Quantiply/rico/tree/master/samza-elasticsearch/src/main/java/org/apache/samza/system/elasticsearch) and will work on creating a patch.

      Attachments

        1. SAMZA-733.patch
          8 kB
          Roger Hoover
        2. SAMZA-733.patch
          6 kB
          Roger Hoover
        3. SAMZA-733.patch
          6 kB
          Roger Hoover
        4. SAMZA-733-WITH-METRICS-BASE-CLASS.patch
          17 kB
          Roger Hoover
        5. SAMZA-733-WITH-METRICS-BASE-CLASS.patch
          17 kB
          Roger Hoover
        6. SAMZA-733-WITH-METRICS-BASE-CLASS.patch
          17 kB
          Roger Hoover

        Activity

          People

            theduderog Roger Hoover
            theduderog Roger Hoover
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: