This was a bug introduced in https://github.com/apache/spark/pull/6928 and affects branch-1.4 and master branch
[Github] Pull Request #7022 (shivaram)