[SPARK-18200] GraphX Invalid initial capacity when running triangleCount

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0, 2.0.1, 2.0.2
• Fix Version/s: 2.0.3, 2.1.0
• Component/s: GraphX
• Labels: graph graphx

Environment

Databricks, Ubuntu 16.04, macOS Sierra

• Target Version/s: 2.0.3, 2.1.0
• Assignee: Dongjoon Hyun
• Reporter: Denny Lee
• Created: 01/Nov/16 15:35
• Updated: 04/Dec/16 02:48
• Resolved: 03/Nov/16 06:49

Description

Running GraphX triangle count on large-ish file results in the "Invalid initial capacity" error when running on Spark 2.0 (tested on Spark 2.0, 2.0.1, and 2.0.2). You can see the results at: http://bit.ly/2eQKWDN

Running the same code on Spark 1.6 and the query completes without any problems: http://bit.ly/2fATO1M

As well, running the GraphFrames version of this code runs as well (Spark 2.0, GraphFrames 0.2): http://bit.ly/2fAS8W8

Reference Stackoverflow question:
Spark GraphX: requirement failed: Invalid initial capacity (http://stackoverflow.com/questions/40337366/spark-graphx-requirement-failed-invalid-initial-capacity)

Issue Links

• links to: [Github] Pull Request #15741 (dongjoon-hyun)
• links to: [Github] Pull Request #15754 (dongjoon-hyun)