Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-1639

Metrics producer does not produce to all partitions

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.9.1
    • Fix Version/s: None
    • Component/s: metrics
    • Labels:
      None
    • Environment:

      Samza: 0.9.1

      Kafka: 0.8.2.2

      Description

      Hi.

       

      I have some tasks that are submitting metrics to a kafka topic, via:

       

      metrics.reporter.snapshot.class=org.apache.samza.metrics.reporter.MetricsSnapshotReporterFactory

      metrics.reporters=snapshot

      metrics.reporter.snapshot.stream=kafka.samza_metrics

      serializers.registry.metrics.class=org.apache.samza.serializers.MetricsSnapshotSerdeFactory

      systems.kafka.streams.samza_metrics.samza.msg.serde=metrics

       

      But we are seeing a weird issue where messages only get sent to a subset of the topic partitions:

       

      ./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic samza_metrics --time -1

      SLF4J: Class path contains multiple SLF4J bindings.

      SLF4J: Found binding in [jar:file:/app/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]

      SLF4J: Found binding in [jar:file:/app/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]

      SLF4J: Found binding in [jar:file:/app/kafka/core/build/dependant-libs-2.10.4/slf4j-simple-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]

      SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

      SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

      samza_metrics:0:20311035

      samza_metrics:1:0

      samza_metrics:2:0

      samza_metrics:3:23723469

      samza_metrics:4:0

      samza_metrics:5:40708920

      samza_metrics:6:0

      samza_metrics:7:0

      samza_metrics:8:10721831

      samza_metrics:9:0

      samza_metrics:10:50921817

      samza_metrics:11:0

      samza_metrics:12:0

      samza_metrics:13:8637095

      samza_metrics:14:11391870

      samza_metrics:15:20299587

      samza_metrics:16:15689619

      samza_metrics:17:0

      samza_metrics:18:6834402

      samza_metrics:19:0

      samza_metrics:20:30792418

      samza_metrics:21:0

      samza_metrics:22:7186731

      samza_metrics:23:5241719

      samza_metrics:24:0

      samza_metrics:25:29958293

      samza_metrics:26:0

      samza_metrics:27:8097351

      samza_metrics:28:13629794

      samza_metrics:29:0

       

      Even of the ones that are being produced to, it's not very balanced.  We don't see this issue with any of of the 'normal' samza input/output topics, or with any of the non-samza related topics we have in this cluster.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              justin.killen Justin
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: