Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-584

LoggingMetricsConsumer metrics.log file is shared by multiple topologies.



    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.6
    • Fix Version/s: 1.0.0, 0.10.1, 2.0.0
    • Component/s: storm-core
    • Labels:


      The current cluster.xml file has special configuration for the LoggingMetricsConsumer to place it in a metrics.log file. There are several issues with this.

      The worker, where the LoggingMetricsConsumer is run, is configured using worker.xml and not cluster.xml.
      The metrics.log file is shared by all workers running on the same node. So if by chance we have LoggingMetricsConsumer instances running at the same time, the resulting log file could have a number of issues, including data corruption.

      This becomes much worse under security where only the first LoggingMetricsConsumer to run on the node will own the file, and all others will fail to log metrics.

      Also there is no way to access metrics.log through the ui now.

      Ideally the metrics.log file will follow a pattern similar to the worker log file, and have the topology name and the worker port a part of it. We also would want to update the logviewer daemon to allow serving up these files and deleting them after a specific period of time. The it would be awesome if the UI could detect that a logging metrics consumer is installed, and provide links to the metrics.log file as well.




            • Assignee:
              jnioche Julien Nioche
              revans2 Robert Joseph Evans
            • Votes:
              1 Vote for this issue
              4 Start watching this issue


              • Created: