Running the following code hangs pyspark in a shell:
It happens in the master branch, but not branch-1.0. And it seems that it only happens in a pyspark shell. Andrew Or helped confirm this bug.
pyspark - RDD action hangs (after previously succeeding)
Bad exception if multiple jars exist when running PySpark