I use different versions of spark to analyze random forest scores..
- spark-core_2.10 and version 2.0.0
- RandomForestsKaggle Score = 0.8978765219058574
- spark-core_2.11 and version 2.4.0
- RandomForestsKaggle Score = 0.8886987035251259
This case is Titanic Competitions on the Kaggle. https://www.kaggle.com/c/titanic
After upgrading the spark version(version 2.4.0), the random forest score dropped(0.01).
Expect random forest score not to drop as the version upgrades.