[STORM-412] Allow users to modify logging levels of running topologies.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: storm-core
• Assignee: Kishor Patil
• Reporter: Robert Joseph Evans
• Created: 17/Jul/14 14:54
• Updated: 05/Oct/15 19:24
• Resolved: 05/Oct/15 19:24

Description

It really would be great to be able to turn on debug logging for different parts of an already running topology, and then turn them off again later. Or even better if they could turn it on for just a few workers, so we don't flood the logs for a very large topology.

logback already has the ability to refresh its config periodically so really what we needs is some code to generate logback configs on the fly based off of the currently desired user settings, and API to modify those dynamically, and a way to distribute those configs/settings to the supervisors. I am not too concerned about persisting these settings long term. If nimbus goes down and they reset back to default, I think that would be OK.

For the distribution of the configs I think it would be best to setup a RESTful web service that is a part of nimbus. logback already supports using http to download configs. The service could use the cached logging settings for a given topology, or individual worker and the URL of the request to generate a logging config on the fly for a specific worker. As for the APIs I think a few new thrift calls to nimbus would be good, a command line tool, and possibly something on the UI.

Issue Links

• is related to: STORM-736 Add a RESTful API to print all of the thread's information and stack traces of Nimbus/Supervisor/Worker Process