I was reviewing TestMRIntermediateDataEncryption. The unit test has actually little to do with encryption.
I have the following conclusion:
- Enabling/Disabling MRJobConfig.MR_ENCRYPTED_INTERMEDIATE_DATA does not change the behavior of the unit test.
- There are no spill files generated by either mappers/reducers
- Wrapping I/O streams with Crypto never happens during the execution of the unit test.
Unless I misunderstand the purpose of that unit test, I suggest that it gets re-implemented so that it validates encryption in spilled intermediate data.