+1 on a feature like that. We need some script/tool/thread that can run major compactions based on load and abort if the load goes over a certain threshold. Once the low load is resumed we can continue where left off. Do this region by region, with a configurable number, i.e. one per cluster, one per node, and so on.
We should also add a JMX/API call that returns the compaction status per server. It should list the various compaction queues, live compactions, their scope, and region/cf they work on. Maybe put this into the ServerInfo?