Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-5341

Push the security 0.92 profile to maven repo

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Won't Fix
    • Affects Version/s: 0.92.1, 0.94.0
    • Fix Version/s: 0.92.3
    • Component/s: build, security
    • Labels:
      None

      Description

      Hbase 0.92.0 was released with two artifacts, plain and security. The security code is built with -Psecurity. There are two tarballs, but only the plain jar in maven repo at repository.a.o.

      I see no reason to do a separate artifact for the security related code, since 0.92 already depends on secure Hadoop 1.0.0, and all of the security related code is not loaded by default. In this issue, I propose, we merge the code under /security to src/ and remove the maven profile.

      Edit: after some discussion, and the plans for modularizing the build to include a security module, we changed the issue description to push the security jars in 0.92.1 to maven repo.

        Issue Links

          Activity

          Hide
          stack stack added a comment -

          Resolving as won't fix. Reopen if I have it wrong. We are a long way from 0.92.

          Show
          stack stack added a comment - Resolving as won't fix. Reopen if I have it wrong. We are a long way from 0.92.
          Hide
          ivarley Ian Varley added a comment -

          Virtual knot is fine (though I'd lobby for an actual one too, for reasons of style). Thanks for the clarification!

          Show
          ivarley Ian Varley added a comment - Virtual knot is fine (though I'd lobby for an actual one too, for reasons of style). Thanks for the clarification!
          Hide
          stack stack added a comment -

          Ian

          The change has been done in trunk – there is one artifact only now rather than one for insecure and another for secure. I was keeping this issue around as reminder that on release of next 0.92/0.94 versions, that we'd be sure to put the -security bundle too as Enis suggests (It might not be possible given dumb mvn release tool but I was going to try it next time around). So this issue is a knot on my finger. Would you like me to make an actual knot on my finger or would you like me open a more specific issue instead?

          Show
          stack stack added a comment - Ian The change has been done in trunk – there is one artifact only now rather than one for insecure and another for secure. I was keeping this issue around as reminder that on release of next 0.92/0.94 versions, that we'd be sure to put the -security bundle too as Enis suggests (It might not be possible given dumb mvn release tool but I was going to try it next time around). So this issue is a knot on my finger. Would you like me to make an actual knot on my finger or would you like me open a more specific issue instead?
          Hide
          ivarley Ian Varley added a comment -

          Is this a done deal now Stack? Looking at:

          http://repo2.maven.org/maven2/org/apache/hbase/hbase/0.92.1/

          I don't see any separate security JARs (not sure if they're supposed to be there or not).

          Show
          ivarley Ian Varley added a comment - Is this a done deal now Stack? Looking at: http://repo2.maven.org/maven2/org/apache/hbase/hbase/0.92.1/ I don't see any separate security JARs (not sure if they're supposed to be there or not).
          Hide
          stack stack added a comment -

          I tried doing this but each time, though it built the security artifacts under the target dir, it'd go upload to nexus the insecures. Here is command I was using:

          % mvn -Papache-release,security release:perform
          

          Need to add in some move switches it seems for release plugin when security profile enabled. Any input?

          Show
          stack stack added a comment - I tried doing this but each time, though it built the security artifacts under the target dir, it'd go upload to nexus the insecures. Here is command I was using: % mvn -Papache-release,security release:perform Need to add in some move switches it seems for release plugin when security profile enabled. Any input?
          Hide
          stack stack added a comment -

          Making blocker on 0.92.1

          Show
          stack stack added a comment - Making blocker on 0.92.1
          Hide
          enis Enis Soztutar added a comment -

          @Jonathan, good idea. I have changed the title of this issue, linked HBASE-4336, and left a comment there. I guess the security module can be built as a part of HBASE-4336 or a subticket there.

          Show
          enis Enis Soztutar added a comment - @Jonathan, good idea. I have changed the title of this issue, linked HBASE-4336 , and left a comment there. I guess the security module can be built as a part of HBASE-4336 or a subticket there.
          Hide
          enis Enis Soztutar added a comment -

          I have recycled this issue, and changed the title.

          Show
          enis Enis Soztutar added a comment - I have recycled this issue, and changed the title.
          Hide
          jmhsieh Jonathan Hsieh added a comment -

          So HBASE-5288 is now resolved – maybe we should rename/create issues for "push the security 0.92 profile to maven repo" and to make a "security submodule (probably 0.94?)"

          Show
          jmhsieh Jonathan Hsieh added a comment - So HBASE-5288 is now resolved – maybe we should rename/create issues for "push the security 0.92 profile to maven repo" and to make a "security submodule (probably 0.94?)"
          Hide
          stack stack added a comment -

          Will do. Let me peg this issue against 0.92.1 so we don't forget it.

          Show
          stack stack added a comment - Will do. Let me peg this issue against 0.92.1 so we don't forget it.
          Hide
          enis Enis Soztutar added a comment -

          thanks Stack, i feel for your pain

          This means that, we should do every 0.92 release with two artifacts. Can you resolve this issue as duplicate, when you push the security maven profile?

          Show
          enis Enis Soztutar added a comment - thanks Stack, i feel for your pain This means that, we should do every 0.92 release with two artifacts. Can you resolve this issue as duplicate, when you push the security maven profile?
          Hide
          stack stack added a comment -

          @Enis 0.92 won't be modularized. HBASE-5288 will be fixed in 0.92.1. Regards getting security artifacts into maven, I'm not sure how I'd do that. I suppose I'd do everything w/ the security profile. Will try it (Thats going to be fun w/ our build taking two hours and mvn rebuilding 4 times I believe per artifact before I the artifact gets hoisted to apache staging).

          Show
          stack stack added a comment - @Enis 0.92 won't be modularized. HBASE-5288 will be fixed in 0.92.1. Regards getting security artifacts into maven, I'm not sure how I'd do that. I suppose I'd do everything w/ the security profile. Will try it (Thats going to be fun w/ our build taking two hours and mvn rebuilding 4 times I believe per artifact before I the artifact gets hoisted to apache staging).
          Hide
          enis Enis Soztutar added a comment -

          I agree with Jonathan, that if the compiled binary works with 0.20.xxx, which as far as I know is the case, then compilation should not be a big concern. And I agree completely if we are going to modularize the build, then hbase-security module makes sense. Do we also plan to backport modules to 0.92 branch. If not, then we should either go with the plan in this jira, or fix HBASE-5288, and push security artifacts to maven repo.

          Show
          enis Enis Soztutar added a comment - I agree with Jonathan, that if the compiled binary works with 0.20.xxx, which as far as I know is the case, then compilation should not be a big concern. And I agree completely if we are going to modularize the build, then hbase-security module makes sense. Do we also plan to backport modules to 0.92 branch. If not, then we should either go with the plan in this jira, or fix HBASE-5288 , and push security artifacts to maven repo.
          Hide
          apurtell Andrew Purtell added a comment -

          If we drop support for 0.20.x then compilation issues mostly go away. It won't fail for lack of security APIs in Hadoop.

          I lean with Gary's view that security belongs in a (Maven) module.

          Show
          apurtell Andrew Purtell added a comment - If we drop support for 0.20.x then compilation issues mostly go away. It won't fail for lack of security APIs in Hadoop. I lean with Gary's view that security belongs in a (Maven) module.
          Hide
          jmhsieh Jonathan Hsieh added a comment -

          I'd be happy if the tarball(s) for 0.92.1 just come out with the ./security directory in the correct place in tarball. I would think that would be the most expedient thing to do. (this is however, likely easier said than done)

          @Gary

          • does compiling against hdfs 1.0.0 and running this new hbase jar in non-secure mode against the append branch hdfs jar work? If it works, I'm not convinced compilation matters as much. I do most of my system testing of these release from jars compiled against hadoop 1.0.0 but running on top of a cdh3 hdfs version – no problems. If the hbase binary doesn't work, then I agree that this a concern – it would block shops locked into their own hdfs branches that don't support hadoop security.
          • using the module approach, the security stuff would be a separate jar that we could add to the classpath right?

          @Ted

          • There are plenty of pieces we've release that been haven't tested by many people. That said, regardless of how included, I can commit that we'll be testing the access control and security features.

          @Stack

          • Hadoop 1.0.0 and CDH3 both have security and append. The next HDFS it seems most folks are coalescing around is 0.23, which also has security and append.
          Show
          jmhsieh Jonathan Hsieh added a comment - I'd be happy if the tarball(s) for 0.92.1 just come out with the ./security directory in the correct place in tarball. I would think that would be the most expedient thing to do. (this is however, likely easier said than done) @Gary does compiling against hdfs 1.0.0 and running this new hbase jar in non-secure mode against the append branch hdfs jar work? If it works, I'm not convinced compilation matters as much. I do most of my system testing of these release from jars compiled against hadoop 1.0.0 but running on top of a cdh3 hdfs version – no problems. If the hbase binary doesn't work, then I agree that this a concern – it would block shops locked into their own hdfs branches that don't support hadoop security. using the module approach, the security stuff would be a separate jar that we could add to the classpath right? @Ted There are plenty of pieces we've release that been haven't tested by many people. That said, regardless of how included, I can commit that we'll be testing the access control and security features. @Stack Hadoop 1.0.0 and CDH3 both have security and append. The next HDFS it seems most folks are coalescing around is 0.23, which also has security and append.
          Hide
          jmhsieh Jonathan Hsieh added a comment -

          This is essentially a dupe of HBASE-5288.

          Show
          jmhsieh Jonathan Hsieh added a comment - This is essentially a dupe of HBASE-5288 .
          Hide
          stack stack added a comment -

          This would break the ability to compile HBase 0.92+ against Hadoop releases without security.

          Perhaps we could entertain breaking this for 0.94.0? i.e. saying we only run on hadoops w/ security? (CDH3 has it? What doesn't that we want to run on by the time 0.94.0 is out).

          On modularization, yes, if hbase-4336 is done soon, security is a natural. Otherwise, we should do as Enis suggests.

          Show
          stack stack added a comment - This would break the ability to compile HBase 0.92+ against Hadoop releases without security. Perhaps we could entertain breaking this for 0.94.0? i.e. saying we only run on hadoops w/ security? (CDH3 has it? What doesn't that we want to run on by the time 0.94.0 is out). On modularization, yes, if hbase-4336 is done soon, security is a natural. Otherwise, we should do as Enis suggests.
          Hide
          ghelmling Gary Helmling added a comment -

          This would break the ability to compile HBase 0.92+ against Hadoop releases without security. Even though we currently compile against 1.0 by default, we haven't block the ability to compile against previous versions. So this would be a change, especially if there's anyone out there running on builds of the 0.20-append branch.

          We've also been discussing moving in the direction of a modular build (see HBASE-4336). The current security/ tree is practically a module already, just lacking it's own pom.xml. Moving security/ back up into src/ would be a step back into the opposite direction, keeping us with a monolithic release. This may make the packaging slightly simpler, but it still won't make it any easier to test out all the combinations of optional components. The security profile does currently use it's own configuration for testing, so at least we get execution of the full test suite using SecureRpcEngine. The full test suite is really overkill for testing. We could use just a good set of RPC focused tests if we had them. But in my opinion that kind of focused testing would be easier to handle in a security module than are part of a monolithic build.

          Show
          ghelmling Gary Helmling added a comment - This would break the ability to compile HBase 0.92+ against Hadoop releases without security. Even though we currently compile against 1.0 by default, we haven't block the ability to compile against previous versions. So this would be a change, especially if there's anyone out there running on builds of the 0.20-append branch. We've also been discussing moving in the direction of a modular build (see HBASE-4336 ). The current security/ tree is practically a module already, just lacking it's own pom.xml. Moving security/ back up into src/ would be a step back into the opposite direction, keeping us with a monolithic release. This may make the packaging slightly simpler, but it still won't make it any easier to test out all the combinations of optional components. The security profile does currently use it's own configuration for testing, so at least we get execution of the full test suite using SecureRpcEngine . The full test suite is really overkill for testing. We could use just a good set of RPC focused tests if we had them. But in my opinion that kind of focused testing would be easier to handle in a security module than are part of a monolithic build.
          Hide
          enis Enis Soztutar added a comment -

          Agreed. Conceptually, security related features are not very different that other features. They can be included in the code base, disabled by default, and marked as experimental if not tested well.

          Show
          enis Enis Soztutar added a comment - Agreed. Conceptually, security related features are not very different that other features. They can be included in the code base, disabled by default, and marked as experimental if not tested well.
          Hide
          zhihyu@ebaysf.com Ted Yu added a comment -

          If you search for the voting process by entering the following in http://search-hadoop.com:
          'ANN: The fifth hbase 0.92.0 release candidate is available for download'
          one can hardly tell whether the voters tested with secure hbase tar ball.

          If we produce one artifact (I think we should), some voters have to test security features before we declare new release.

          Show
          zhihyu@ebaysf.com Ted Yu added a comment - If you search for the voting process by entering the following in http://search-hadoop.com: 'ANN: The fifth hbase 0.92.0 release candidate is available for download' one can hardly tell whether the voters tested with secure hbase tar ball. If we produce one artifact (I think we should), some voters have to test security features before we declare new release.
          Hide
          enis Enis Soztutar added a comment -

          I don't see a reason for changing the release process. The vote for 0.92.0 release included both the plain and secure artifacts, see http://comments.gmane.org/gmane.comp.java.hadoop.hbase.devel/25671

          Show
          enis Enis Soztutar added a comment - I don't see a reason for changing the release process. The vote for 0.92.0 release included both the plain and secure artifacts, see http://comments.gmane.org/gmane.comp.java.hadoop.hbase.devel/25671
          Hide
          zhihyu@ebaysf.com Ted Yu added a comment -

          Yes.

          Show
          zhihyu@ebaysf.com Ted Yu added a comment - Yes.
          Hide
          enis Enis Soztutar added a comment -

          Sorry, I did not understand what you are referring to with the certification process. Do you mean voting for the RC, signing the release, etc?

          Show
          enis Enis Soztutar added a comment - Sorry, I did not understand what you are referring to with the certification process. Do you mean voting for the RC, signing the release, etc?
          Hide
          zhihyu@ebaysf.com Ted Yu added a comment -

          The certification for 0.92.0 was for insecure HBase artifact.
          If we only produce one secure artifact, would certification process change ?

          Show
          zhihyu@ebaysf.com Ted Yu added a comment - The certification for 0.92.0 was for insecure HBase artifact. If we only produce one secure artifact, would certification process change ?
          Hide
          enis Enis Soztutar added a comment -

          The only artifact build will be plain 0.92.1 or 0.94 (no -security appended). But this will include security related code. It's like Hadoop-1.0.0 which includes security related codes in one artifact.

          Show
          enis Enis Soztutar added a comment - The only artifact build will be plain 0.92.1 or 0.94 (no -security appended). But this will include security related code. It's like Hadoop-1.0.0 which includes security related codes in one artifact.
          Hide
          zhihyu@ebaysf.com Ted Yu added a comment -

          If we remove the remove the maven 'security' profile, only secure HBase artifacts would be built, right ?
          Since most users wouldn't be using secure HBase features, I think this might introduce confusion for them.

          Show
          zhihyu@ebaysf.com Ted Yu added a comment - If we remove the remove the maven 'security' profile, only secure HBase artifacts would be built, right ? Since most users wouldn't be using secure HBase features, I think this might introduce confusion for them.
          Hide
          enis Enis Soztutar added a comment -

          Also, there is no secure artifact in the maven repo, so depending on when 0.92.1 is cut out, we might want to push 0.92.0-security as well.

          Show
          enis Enis Soztutar added a comment - Also, there is no secure artifact in the maven repo, so depending on when 0.92.1 is cut out, we might want to push 0.92.0-security as well.
          Hide
          enis Enis Soztutar added a comment -

          I can provide a patch, if we agree on this.

          Show
          enis Enis Soztutar added a comment - I can provide a patch, if we agree on this.

            People

            • Assignee:
              enis Enis Soztutar
              Reporter:
              enis Enis Soztutar
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development