Normal Description
We are using LCS and have total of 38000 SSTables for one CF. During LCS, there could be over a thousand SSTable involved. All those SSTable IDs are stored in ancestors field of SSTableMetatdata for the new table. In our case, it consumes more than 1G of heap memory for those field. Put it in perspective, the ancestors consume 2 - 3 times more memory than bloomfilter (fp = 0.1 by default) in LCS.
We should remove those ancestors from SSTableMetadata after the compaction is finished and the old SSTable is removed. It might be a big deal for Sized Compaction since there are small number of SSTable involved. But it consumes a lot of memory for LCS.
At least, we shouldn't load those ancestors to the memory during startup if the files are removed.
I would love to contribute and provide patch. Please let me know how to start.
