[SPARK-28513] Compute distinct label sets instead of subsets

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.1.0
• Component/s: Graph
• Epic Link: Property Graph / Cypher / Algorithms
• Assignee: Unassigned
• Reporter: Martin Junghanns
• Created: 25/Jul/19 10:24
• Updated: 16/Mar/20 22:53

Description

CypherSession::createDataFrame(nodes: DataFrame, rels: DataFrame)

currently computes NodeFrames by filtering label columns, computing all possible subsets and creating one NodeFrame for each subset. This results in 2^n sets / NodeFrames.

Instead, we should compute the distinct label sets that actually occur in the nodes DataFrame.