Details

    • Type: Sub-task Sub-task
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.0.0
    • Fix Version/s: None
    • Component/s: security
    • Labels:
    • Target Version/s:

      Description

      The work defined here is to extend the Compression Codec framework to the encryption framework for handling encryption and decryption specific requirements in Hadoop.
      The targets of the encryption framework are:
      1. Establishes a common abstraction of the API level that can be shared by all crypto codec implementations as well as users that use the API.
      2. Provides a foundation for other components in Hadoop such as Map Reduce or HBase to support encryption features.

      The design document is available in the HADOOP-9331.

      1. HADOOP-9333.patch
        93 kB
        Jerry Chen
      2. HADOOP-9333.patch
        92 kB
        Jerry Chen

        Issue Links

          Activity

          Hide
          Jerry Chen added a comment -

          Patch attached.

          Show
          Jerry Chen added a comment - Patch attached.
          Hide
          Jerry Chen added a comment -

          The patched submitted. Please help review. Comments are welcomed.

          Show
          Jerry Chen added a comment - The patched submitted. Please help review. Comments are welcomed.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12570715/HADOOP-9333.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          -1 one of tests included doesn't have a timeout.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2227//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/2227//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2227//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570715/HADOOP-9333.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. -1 one of tests included doesn't have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2227//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/2227//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2227//console This message is automatically generated.
          Hide
          Jerry Chen added a comment -

          New patch that fixed the findbugs and adding timeout in test.

          Show
          Jerry Chen added a comment - New patch that fixed the findbugs and adding timeout in test.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12570744/HADOOP-9333.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 tests included appear to have a timeout.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2228//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2228//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570744/HADOOP-9333.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 tests included appear to have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2228//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2228//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          Jerry, I did not look at the patch in detail. At first glance it seems you'll be able to use a single codec, how are you planning to support different compression/encryption codec combinations? Or it is already there?

          Show
          Alejandro Abdelnur added a comment - Jerry, I did not look at the patch in detail. At first glance it seems you'll be able to use a single codec, how are you planning to support different compression/encryption codec combinations? Or it is already there?
          Hide
          Jerry Chen added a comment -

          Alejandro Abdelnur, The crypto codec provide high level abstraction related to a codec that needs a crypto context. When considering the case that we do compression before the encryption, a compression codec can be configured as part of the crypto codec configuration and let the crypto codec implemenation handling the compression by using the specified compression codec and then do encryption after that. And the decryption process is reverse.

          Show
          Jerry Chen added a comment - Alejandro Abdelnur , The crypto codec provide high level abstraction related to a codec that needs a crypto context. When considering the case that we do compression before the encryption, a compression codec can be configured as part of the crypto codec configuration and let the crypto codec implemenation handling the compression by using the specified compression codec and then do encryption after that. And the decryption process is reverse.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12570744/HADOOP-9333.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3700//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3700//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570744/HADOOP-9333.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3700//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3700//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 32s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
          +1 javac 7m 32s There were no new javac warning messages.
          +1 javadoc 9m 32s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 4s The applied patch generated 196 new checkstyle issues (total was 317, now 511).
          -1 whitespace 0m 1s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 1m 41s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 common tests 22m 46s Tests passed in hadoop-common.
              59m 39s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12570744/HADOOP-9333.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 6ae2a0d
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6426/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6426/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6426/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6426/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6426/console

          This message was automatically generated.

          Show
          Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 32s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 32s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 4s The applied patch generated 196 new checkstyle issues (total was 317, now 511). -1 whitespace 0m 1s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 1m 41s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 22m 46s Tests passed in hadoop-common.     59m 39s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12570744/HADOOP-9333.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6ae2a0d checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6426/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6426/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6426/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6426/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6426/console This message was automatically generated.

            People

            • Assignee:
              Unassigned
              Reporter:
              Jerry Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              21 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 168h
                168h
                Remaining:
                Remaining Estimate - 168h
                168h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development