[CASSANDRA-5555] Allow sstableloader to handle a larger number of files - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Normal
Resolution: Fixed
Fix Version/s: 1.2.7, 2.0 beta 2
Component/s: Legacy/Tools
Labels:
None

Description

With the default heap size, sstableloader will OOM when there are roughly 25k files in the directory to load. It's easy to reach this number of files in a single LCS column family.

By avoiding creating all SSTableReaders up front in SSTableLoader, we should be able to increase the number of files that sstableloader can handle considerably.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

5555-01.txt
24/May/13 03:49
16 kB
David Brosius
5555-02.txt
31/May/13 03:37
14 kB
David Brosius
5555-2.txt
19/Jun/13 13:22
12 kB
Jonathan Ellis
5555-fix-heap-and-streaming-1.2.patch
11/Jul/13 20:28
35 kB
Tom Hobbs
5555-fix-heap-and-streaming-1.2-v2.patch
12/Jul/13 16:12
35 kB
Tom Hobbs
5555-fix-heap-and-streaming-trunk.patch
18/Jul/13 19:23
34 kB
Tom Hobbs
cass_5555_pic_8.png
19/Jun/13 05:29
260 kB
Alex Zarutin
CASSANDRA-5555.txt
19/Jun/13 05:52
11 kB
Alex Zarutin
CASSANDRA-5555.txt
10/Jun/13 20:14
9 kB
Alex Zarutin
CASSANDRA-5555.txt
10/Jun/13 19:33
10 kB
Alex Zarutin

Issue Links

is related to

CASSANDRA-5653 NoSuchElementException using sstableloader when extra files (not Index/Data) exist under keyspace/CF of the restored folder

Resolved

Activity

People

Assignee:: Tom Hobbs

Reporter:: Tom Hobbs

Authors:: Tom Hobbs

Reviewers:: Jonathan Ellis

Tester:: Alex Zarutin

Votes:: 0 Vote for this issue

Watchers:: 7 Start watching this issue

Dates

Created:: 10/May/13 16:07

Updated:: 16/Apr/19 09:32

Resolved:: 18/Jul/13 20:15