[CASSANDRA-674] New SSTable Format

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Won't Fix
• Assignee: Stu Hood Assign to me
• Reporter: Stu Hood

Authors

Stu Hood

• Created: 06/Jan/10 10:33
• Updated: 16/Apr/19 12:33
• Resolved: 20/Sep/13 19:42

Description

Various tickets exist due to limitations in the SSTable file format, including #16, #47 and #328. Attached is a proposed design/implementation of a new file format for SSTables that addresses a few of these limitations.

This v2 implementation is not ready for serious use: see comments for remaining issues. It is roughly the format described here: http://wiki.apache.org/cassandra/FileFormatDesignDoc

Issue Links

• blocks: CASSANDRA-598 SuperColumns need to be indexed
• blocks: CASSANDRA-2319 Promote row index
• blocks: CASSANDRA-494 add remove_slice to the api
• is blocked by: CASSANDRA-2398 Type specific compression
• is blocked by: CASSANDRA-3067 Simple SSTable Pluggability
• is blocked by: CASSANDRA-2576 Rewrite into new file post streaming
• is blocked by: CASSANDRA-2629 Move key reads into SSTableIterators
• is blocked by: CASSANDRA-389 SSTable Versioning
• is blocked by: CASSANDRA-2062 Better control of iterator consumption
• is blocked by: CASSANDRA-2144 Don't slurp rows into memory during export
• is blocked by: CASSANDRA-2145 Simplify ColumnSortedMap
• is depended upon by: CASSANDRA-808 Need a way to skip corrupted data in SSTables
• is related to: AVRO-757 Allow a DataFile to be reopened without reading the head