[HIVE-1040] use sed rather than diff for masking out noise in diff-based tests

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.4.1
• Fix Version/s: 0.9.0
• Component/s: Testing Infrastructure
• Hadoop Flags: Reviewed
• Assignee: Marek Sapota
• Reporter: John Sichi
• Created: 11/Jan/10 21:32
• Updated: 30/Apr/12 21:11
• Resolved: 14/Dec/11 01:01

Description

The current diff -I approach has two problems: (1) it does not allow resolution finer than line-level, so it's impossible to mask out pattern occurrences within a line, and (2) it produces unmasked files, so if you run diff on the command line to compare the result .q.out with the checked-in file, you see the noise.

My suggestion is to first run sed to replace noise patterns with an unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files without using any -I.

This would require a one-time hit to update all existing .q.out files so that they would contain the pre-masked results.

Issue Links

• relates to: HIVE-2834 Diff masking it too aggressive in index_bitmap*.q and index_compact*.q tests