Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2568

'api/vi/topology/:id/lag' returns empty json {}

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.1.1, 1.2.0
    • Component/s: storm-core
    • Labels:
      None

      Description

      Hello

      I've tried to use storm-kafka-monitor, and it works fine on command line If I changed 'toollib/storm-kafka-monitor-*.jar' to 'toollib/storm-kafka-monitor-1.1.0.jar'.

      {"my-kafka-topic-name":{"0":{"consumerCommittedOffset": 74804998, "logHeadOffset": 74805483, "lag": 485},"1":{"consumerCommittedOffset": 74804998, "logHeadOffset": 74805485, "lag": 487},"2":{"consumerCommittedOffset": 74804995, "logHeadOffset": 74805485, "lag": 490},"3":{"consumerCommittedOffset": 74805001, "logHeadOffset": 74805488, "lag": 487},"4":{"consumerCommittedOffset": 74805011, "logHeadOffset": 74805484, "lag": 473},"5":{"consumerCommittedOffset": 74805009, "logHeadOffset": 74805485, "lag": 476},"6":{"consumerCommittedOffset": 74805008, "logHeadOffset": 74805483, "lag": 475},"7":{"consumerCommittedOffset": 74805010, "logHeadOffset": 74805484, "lag": 474},"8":{"consumerCommittedOffset": 73641446, "logHeadOffset": 74805488, "lag": 1164042},"9":{"consumerCommittedOffset": 73641448, "logHeadOffset": 74805489, "lag": 1164041},"10":{"consumerCommittedOffset": 73641443, "logHeadOffset": 74805483, "lag": 1164040},"11":{"consumerCommittedOffset": 73641445, "logHeadOffset": 74805487, "lag": 1164042},"12":{"consumerCommittedOffset": 74805003, "logHeadOffset": 74805486, "lag": 483},"13":{"consumerCommittedOffset": 74804999, "logHeadOffset": 74805482, "lag": 483},"14":{"consumerCommittedOffset": 74805002, "logHeadOffset": 74805483, "lag": 481},"15":{"consumerCommittedOffset": 74805002, "logHeadOffset": 74805484, "lag": 482},"16":{"consumerCommittedOffset": 74804994, "logHeadOffset": 74805482, "lag": 488},"17":{"consumerCommittedOffset": 74805002, "logHeadOffset": 74805489, "lag": 487},"18":{"consumerCommittedOffset": 74805003, "logHeadOffset": 74805488, "lag": 485},"19":{"consumerCommittedOffset": 74805003, "logHeadOffset": 74805489, "lag": 486}}}
      

      but it gives empty result when I call below api.

      /api/v1/topology/:id/lag
      ...
      
      {
      "MySpoutName": {
      "spoutLagResult": {},
      "spoutId": "MySpoutName",
      "spoutType": "KAFKA"
      }
      }
      

      I think that needs to fix "groupid" to "group.id" in TopologySpoutLag.java I debug it, but groupid is right.

      the reason was topics has square brackets in command.

      2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] json configuration: {config.security.protocol=null, config.bootstrap.servers=kafka.xxx.com:9092, config.topics=[my-kafka-topic], config.groupid=my-storm-kafka-spout-groupid, topology.tasks=5}
      
      2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] /my/program/storm/bin/storm-kafka-monitor
      2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] -t
      2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] [my-kafka-topic]
      2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] -g
      2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] my-storm-kafka-spout-groupid
      2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] -b
      2017-06-23 19:55:56.725 o.a.s.u.TopologySpoutLag qtp426435961-51 [INFO] kafka.xxx.com:9092
      

      the square brackets automatically added because of this

      package org.apache.storm.kafka.spout;
      public class NamedSubscription extends Subscription {
      ...
          @Override
          public String getTopicsString() {
              return String.valueOf(topics);
          }
      

      topics is Collections. so String.valueOf returns value with square brackets.

      I fixed the code that remove square brackets in TopologySpoutLag.java for my case.
      but I think that fixing 'getTopicsString of NamedSubscription.java in org.apache.storm.kafka.spout' is might be better.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                haewanj Haewan Jeong
                Reporter:
                haewanj Haewan Jeong
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1.5h
                  1.5h