Details
-
Improvement
-
Status: Resolved
-
Normal
-
Resolution: Fixed
-
None
-
Docs
Description
Cassandra saves a sample of IndexInfo objects that store the offset within each partition of every 64KB (by default) range of rows. To find a row, we binary search this sample, then scan the partition of the appropriate range.
The problem is that this scales poorly as partitions grow: on a cache miss, we deserialize the entire set of IndexInfo, which both creates a lot of GC overhead (as noted in CASSANDRA-9754) but is also non-negligible i/o activity (relative to reading a single 64KB row range) as partitions get truly large.
We introduced an "offset map" in CASSANDRA-10314 that allows us to perform the IndexInfo bsearch while only deserializing IndexInfo that we need to compare against, i.e. log(N) deserializations.
Attachments
Attachments
Issue Links
- blocks
-
CASSANDRA-11369 Re-use IndexedEntry / IndexInfo after 11206
- Open
- is depended upon by
-
CASSANDRA-9738 Migrate key-cache to be fully off-heap
- Resolved
-
CASSANDRA-11584 Add stats about index-entries to per sstable-stats
- Open
- is related to
-
CASSANDRA-19557 ShallowIndexedEntry scenario: the same IndexInfo is read multiple times, per every read row
- Resolved
- relates to
-
CASSANDRA-11765 dtest failure in upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_Skips_3_0_x_EndsAt_Trunk_HEAD.rolling_upgrade_test
- Resolved
-
CASSANDRA-9754 Make index info heap friendly for large CQL partitions
- Triage Needed
- supercedes
-
CASSANDRA-10320 Build off-heap key-cache compatible API
- Resolved
-
CASSANDRA-16904 Check if size of object being added to RowCache and KeyCache is bigger than cache capacity
- Resolved