[TINKERPOP-3074] The sample() step is largely unusable with large graphs

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.6.7, 3.7.2
• Component/s: process
• Assignee: Unassigned
• Reporter: Kelvin Lawrence
• Created: 22/Apr/24 16:37
• Updated: 30/Aug/24 16:29

Description

While the `sample` step can be useful with smallish sized amounts of data for random walks and similar, its current implementation makes it unusable with large graphs if you are looking to sample, say, one node, from a graph with a millions or billions of nodes in it.

// This generally works assuming the out() step yields limited numbers of nodes
g.V(1).out().sample(1).out().sample(1) //etc

// This fails for a large graph, usually with an OOM error
g.V().sample(1)

The current implementation of sample() is quite naive and assumes it can fetch everything into memory before computing a result. I have seen many users wanting to start a walk from a random place, and they always try to do g.V().sample(1) or g.E().sample(1) types of queries.