Minor Description
When application-level iterators deep copy the source iterator in a major compaction, the stats can be significantly off. We count two things in a major compaction:
1. Entries read. This is done using a counting iterator sitting just above the system iterators.
2. Entries written. This is done by counting the entries that are written to the RFile.
Here's an example of what we see in the Accumulo logs:
2013-09-06 11:53:31,371 [tabletserver.Compactor] DEBUG: Compaction k;row11;row10 20 read | 382,629 written | 3 entries/sec | 5.337 secs
In this case, we're only counting 20 entries read, presumably because the iterators have been deep copied and the counting iterator that is being polled does not get a complete view of how many entries were read. Instead of 3 entries/sec we should have registered close to 72k entries/sec.
To fix this, should we be counting all reads coming from any of the deep copies of the source iterators? This could be done by using a CountingIterator that keeps one counter for all deep copies. Thread-level counters could be used for lock-free counts in case multiple threads are ever used.