[SPARK-9211] HiveComparisonTest generates incorrect file name for golden answer files on Windows

• Type: Test
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4.1
• Fix Version/s: 1.5.0
• Component/s: SQL, Windows
• Labels: hive sql test windows

Environment

Windows

• Assignee: Christian Kadner
• Reporter: Christian Kadner
• Shepherd: Michael Armbrust
• Created: 21/Jul/15 06:03
• Updated: 03/Sep/15 19:54
• Resolved: 06/Aug/15 21:16

Description

The names of the golden answer files for the Hive test cases (test suites based on HiveComparisonTest) are generated using an MD5 hash of the query text. When the query text contains line breaks then the generated MD5 hash differs between Windows and Linux/OSX (\r\n vs \n).

This results in erroneously created golden answer files from just running a Hive comparison test and makes it impossible to modify or add new test cases with correctly named golden answer files on Windows.

Issue Links

• is duplicated by: SPARK-9274 Tests using golden files can fail depending on new line characters in queries
• links to: [Github] Pull Request #7563 (ckadner)