The MetricsHistoryHandler should disable itself better than it does. It will create a SolrRrdBackendFactory (which has an ExecutorService / thread pool) when it should not when the plugin is disabled. Furthermore, I think this should be disabled by default in tests, as it's excessive machinery that don't matter to 99.9% of tests. This will help our tests run more efficiently.
While working on other issues, I also occasionally saw a thread leak failure with the RRD being the leaked threads. I see that in "close", we don't wait for the executor to actually shut down but it should.