You're right that if a task throws an exception, I think that it will probably not set the counter. I could put the increment in a finally block and that would fix that issue.
Based on a quick look at the code in Counters and Counter, I think that there wouldn't be major issues with thread safety or performance (every operation on a Counter is already synchronized). An extra thread is pretty heavy-weight, though. I think something like this should actually go into the TaskReporter itself; it could just increment the gc counter itself right before sending a status every 3 seconds.
I ran a quick benchmark test which got a handle to the GarbageCollectorMXBean list and polled them; I put all of this in a loop and ran it a million times in about a half-second, so I don't think this would negatively impact performance.