[CASSANDRA-12731] Remove IndexInfo cache from FileIndexInfoRetriever.

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 3.10
• Component/s: Legacy/Local Write-Read Paths
• Assignee: Yasuharu Goto
• Reporter: Yasuharu Goto

Authors

Yasuharu Goto

Reviewers

Robert Stupp

• Created: 29/Sep/16 16:26
• Updated: 16/Apr/19 09:30
• Resolved: 04/Oct/16 13:15

Description

Hi guys.
In the patch of CASSANDRA-11206 , I found that FileIndexInfoRetriever allocates a (potentially very large) IndexInfo array (up to the number of IndexInfo in the RowIndexEntry has) as a cache in every single read path.

After some experiments using LargePartitionTest on my MacBook, I got results that show that removing FileIndexInfoRetriever improves the performance for large partitions like below (latencies reduced by 41% and by 45%).

// LargePartitionsTest.test_13_4G with cache by array
INFO  [main] 2016-09-29 23:11:25,763 ?:? - SELECTs 1 for part=4194304k total=16384M took 94197 ms
INFO  [main] 2016-09-29 23:12:50,914 ?:? - SELECTs 2 for part=4194304k total=16384M took 85151 ms

// LargePartitionsTest.test_13_4G without cache
INFO  [main] 2016-09-30 00:13:26,050 ?:? - SELECTs 1 for part=4194304k total=16384M took 55112 ms
INFO  [main] 2016-09-30 00:14:12,132 ?:? - SELECTs 2 for part=4194304k total=16384M took 46082 ms

Code is here (based on trunk)

Heap memory usage during running LargePartitionsTest (except for 8G test) with array cache(original)

Heap memory usage during running LargePartitionsTest (except for 8G test) without cache

Of course, I have attempted to use some collection containers instead of a plain array. But I could not recognize great improvement enough to justify using these cache mechanism by them. (Unless I did some mistake or overlook about this test)

LargePartitionsTest.test_12_2G	SELECTs 1 (ms)	SELECTs 2 (ms)	Scan (ms)
Original (array)	62736	48562	41540
ConcurrentHashMap	47597	30854	18271
ConcurrentHashMap 2nd trial	44036	26895	17443
LinkedHashCache (capacity=16, limit=10, fifo) 1st	42668	32165	17323
LinkedHashCache (capacity=16, limit=10, fifo) 2nd	48863	28066	18053
LinkedHashCache (capacity=16, limit=16, fifo)	46979	29810	18620
LinkedHashCache (capacity=16, limit=10, lru)	46456	29749	20311
No Cache	47579	32480	18337
No Cache 2nd trial	46534	27670	18700

Code that I used for this comparison is here. LinkedHashCache is a simple fifo/lru cache that is extended by LinkedHashMap.
Scan is a execution time to iterate through the large partition.

So, in this issue, I'd like to propose to remove IndexInfo cache from FileIndexInfoRetriever to improve the performance on large partitions.