Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-1136

Provide a bin script to check consumer lag from KafkaSpout to Kafka topic offsets and integrate with storm ui

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      We store kafkaspout offsets in zkroot + id path in zookeeper. Kafka provides a utility and a protocol request to fetch latest offsets into topic

      example:
      bin/kafka-run-classh.sh kafka.tools.GetOffsetTool 
      

      we should provide a way for the user to check how far the kafka spout read into topic and whats the lag. If we can expose this via UI even better.

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              pshah Parag Shah
              Reporter:
              sriharsha Harsha

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 40m
                40m

                  Issue deployment