Details

    • Type: Sub-task Sub-task
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.0.0
    • Fix Version/s: None
    • Component/s: security
    • Labels:
      None
    • Target Version/s:

      Description

      The work defined here is to extend the Compression Codec framework to the encryption framework for handling encryption and decryption specific requirements in Hadoop.
      The targets of the encryption framework are:
      1. Establishes a common abstraction of the API level that can be shared by all crypto codec implementations as well as users that use the API.
      2. Provides a foundation for other components in Hadoop such as Map Reduce or HBase to support encryption features.

      The design document is available in the HADOOP-9331.

      1. HADOOP-9333.patch
        93 kB
        Jerry Chen
      2. HADOOP-9333.patch
        92 kB
        Jerry Chen

        Issue Links

          Activity

          Hide
          Jerry Chen added a comment -

          Patch attached.

          Show
          Jerry Chen added a comment - Patch attached.
          Hide
          Jerry Chen added a comment -

          The patched submitted. Please help review. Comments are welcomed.

          Show
          Jerry Chen added a comment - The patched submitted. Please help review. Comments are welcomed.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12570715/HADOOP-9333.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          -1 one of tests included doesn't have a timeout.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2227//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/2227//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2227//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570715/HADOOP-9333.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. -1 one of tests included doesn't have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2227//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/2227//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2227//console This message is automatically generated.
          Hide
          Jerry Chen added a comment -

          New patch that fixed the findbugs and adding timeout in test.

          Show
          Jerry Chen added a comment - New patch that fixed the findbugs and adding timeout in test.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12570744/HADOOP-9333.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 tests included appear to have a timeout.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2228//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2228//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570744/HADOOP-9333.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 tests included appear to have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2228//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2228//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          Jerry, I did not look at the patch in detail. At first glance it seems you'll be able to use a single codec, how are you planning to support different compression/encryption codec combinations? Or it is already there?

          Show
          Alejandro Abdelnur added a comment - Jerry, I did not look at the patch in detail. At first glance it seems you'll be able to use a single codec, how are you planning to support different compression/encryption codec combinations? Or it is already there?
          Hide
          Jerry Chen added a comment -

          Alejandro Abdelnur, The crypto codec provide high level abstraction related to a codec that needs a crypto context. When considering the case that we do compression before the encryption, a compression codec can be configured as part of the crypto codec configuration and let the crypto codec implemenation handling the compression by using the specified compression codec and then do encryption after that. And the decryption process is reverse.

          Show
          Jerry Chen added a comment - Alejandro Abdelnur , The crypto codec provide high level abstraction related to a codec that needs a crypto context. When considering the case that we do compression before the encryption, a compression codec can be configured as part of the crypto codec configuration and let the crypto codec implemenation handling the compression by using the specified compression codec and then do encryption after that. And the decryption process is reverse.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12570744/HADOOP-9333.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3700//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3700//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570744/HADOOP-9333.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3700//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3700//console This message is automatically generated.

            People

            • Assignee:
              Unassigned
              Reporter:
              Jerry Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 168h
                168h
                Remaining:
                Remaining Estimate - 168h
                168h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development