Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7322

revisiting TestMRIntermediateDataEncryption

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      I was reviewing TestMRIntermediateDataEncryption. The unit test has actually little to do with encryption.
      I have the following conclusion:

      • Enabling/Disabling MRJobConfig.MR_ENCRYPTED_INTERMEDIATE_DATA does not change the behavior of the unit test.
      • There are no spill files generated by either mappers/reducers
      • Wrapping I/O streams with Crypto never happens during the execution of the unit test.

      Unless I misunderstand the purpose of that unit test, I suggest that it gets re-implemented so that it validates encryption in spilled intermediate data.

        Attachments

        1. MAPREDUCE-7322.001.patch
          89 kB
          Ahmed Hussein
        2. MAPREDUCE-7322.002.patch
          84 kB
          Ahmed Hussein
        3. MAPREDUCE-7322.003.patch
          88 kB
          Ahmed Hussein
        4. MAPREDUCE-7322.004.patch
          95 kB
          Ahmed Hussein
        5. MAPREDUCE-7322.005.patch
          95 kB
          Ahmed Hussein
        6. MAPREDUCE-7322.006.patch
          105 kB
          Ahmed Hussein
        7. MAPREDUCE-7322.007.patch
          92 kB
          Ahmed Hussein
        8. MAPREDUCE-7322.008.patch
          94 kB
          Ahmed Hussein
        9. MAPREDUCE-7322.009.patch
          94 kB
          Ahmed Hussein
        10. MAPREDUCE-7322.branch-2.10.009.patch
          95 kB
          Ahmed Hussein
        11. MAPREDUCE-7322.branch-3.2.009.patch
          94 kB
          Ahmed Hussein

        Issue Links

          Activity

            People

            • Assignee:
              ahussein Ahmed Hussein
              Reporter:
              ahussein Ahmed Hussein

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment