[MAPREDUCE-7322] revisiting TestMRIntermediateDataEncryption

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.4.0, 3.3.1, 2.10.2, 3.2.3
• Component/s: job submission, security, test
• Labels: patch-available
• Assignee: Ahmed Hussein
• Reporter: Ahmed Hussein
• Created: 20/Feb/21 01:11
• Updated: 10/Jun/21 08:10
• Resolved: 16/Mar/21 19:53

Description

I was reviewing TestMRIntermediateDataEncryption. The unit test has actually little to do with encryption.
I have the following conclusion:

• Enabling/Disabling MRJobConfig.MR_ENCRYPTED_INTERMEDIATE_DATA does not change the behavior of the unit test.
• There are no spill files generated by either mappers/reducers
• Wrapping I/O streams with Crypto never happens during the execution of the unit test.

Unless I misunderstand the purpose of that unit test, I suggest that it gets re-implemented so that it validates encryption in spilled intermediate data.

Issue Links

• fixes: MAPREDUCE-7321 TestMRIntermediateDataEncryption does not cleanup data folders
• relates to: MAPREDUCE-7325 Intermediate data encryption is broken in LocalJobRunner