We should upgrade our snappy-java dependency to 184.108.40.206 in order to include a fix for a bug that resulted in worse compression (see https://github.com/xerial/snappy-java/issues/100). I believe that this may partially fix
SPARK-5081, an issue where the size of shuffle data increased from Spark 1.1.x to 1.2.0.
- is duplicated by
SPARK-5081 Shuffle write increases
- links to