In make-distribution.sh, SPARK_HADOOP_VERSION is determined by --hadoop, and SPARK_HIVE is determined by --with-hive, but these two command line options are deprecated by `
Dhadoop.version`/`-Phadoop??` and `-Phive` respectively. The result is that:
- we may end up with something like spark-1.1.0-SNAPSHOT-bin-.tgz unless --name is specified explicitly
- Datanucleus jars will not be included in the distribution folder/tarball unless we set SPARK_HIVE=true explicitly
A possible fix is to check hadoop.version and project.activeProfiles with mvn help:evaluate, similar to what we do now to determine VERSION.