[TINKERPOP-1177] Improve documentation around Spark's storage levels

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.1-incubating
• Fix Version/s: 3.2.0-incubating, 3.1.2-incubating
• Component/s: documentation
• Assignee: Daniel Kuppitz
• Reporter: Daniel Kuppitz
• Created: 25/Feb/16 23:25
• Updated: 31/Mar/16 16:34
• Resolved: 31/Mar/16 16:34

Description

We should add a big warning regarding the two storage level settings, especially since we have MEMORY_ONLY as the default value. What's the impact of changing any of these settings? What is kept in memory, what not? How much memory will my executors need when I use one or another storage level?

Currently these 2 settings mean nothing to the average user (you'll probably only know how to use them, if you have a deep understanding of how Spark works). For the averaage user we should probably also consider to make DISK_ONLY (or perhaps MEMORY_AND_DISK?) the default.