Details

    • Type: Task Task
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.3.0
    • Fix Version/s: 3.5.2, 3.6.0
    • Component/s: build
    • Labels:
      None

      Description

      Remove rpm/deb/... packaging from our source repo. Now that BigTop is available and fully supporting ZK it's no longer necessary for us to attempt to include this.

        Issue Links

          Activity

          Hide
          Johan Hillertz added a comment -

          As part of https://issues.apache.org/jira/browse/ZOOKEEPER-1708 i got pointed to this issue. I can not find the latest 3.4.5 in BigTop the latest version i find is 3.4.3. How would the BigTop be updated?

          If it is not updated at all release it can be good to keep the possibility to make a local package build.

          Show
          Johan Hillertz added a comment - As part of https://issues.apache.org/jira/browse/ZOOKEEPER-1708 i got pointed to this issue. I can not find the latest 3.4.5 in BigTop the latest version i find is 3.4.3. How would the BigTop be updated? If it is not updated at all release it can be good to keep the possibility to make a local package build.
          Hide
          Ralph Tice added a comment -

          -1 for this ticket and +1 for keeping packaging local to the ZK project. I would be interested in helping with the infrastructure work for automated testing of the debs/rpms.

          Show
          Ralph Tice added a comment - -1 for this ticket and +1 for keeping packaging local to the ZK project. I would be interested in helping with the infrastructure work for automated testing of the debs/rpms.
          Hide
          Michi Mutsuzaki added a comment -

          What's wrong with using BigTop?

          Show
          Michi Mutsuzaki added a comment - What's wrong with using BigTop?
          Hide
          Ralph Tice added a comment -

          I don't have anything against BigTop but the path to getting latest version of ZooKeeper packaged should be kept as simple as possible. Let BigTop mirror the packaging and merge in their own patches if there are Hadoop-specific use cases to support, but ideally the packaging in the ZooKeeper repository should be made generic enough to support any Hadoop use case and BigTop should simply consume the same packaging from upstream. IMO.

          Show
          Ralph Tice added a comment - I don't have anything against BigTop but the path to getting latest version of ZooKeeper packaged should be kept as simple as possible. Let BigTop mirror the packaging and merge in their own patches if there are Hadoop-specific use cases to support, but ideally the packaging in the ZooKeeper repository should be made generic enough to support any Hadoop use case and BigTop should simply consume the same packaging from upstream. IMO.
          Hide
          Michi Mutsuzaki added a comment -

          Ok let's hear from other people to what their thoughts are.

          By the way, we don't actively monitor the github mirror for pull requests. Please follow the steps described here to submit your patches.

          http://wiki.apache.org/hadoop/ZooKeeper/HowToContribute

          Thanks!
          --Michi

          Show
          Michi Mutsuzaki added a comment - Ok let's hear from other people to what their thoughts are. By the way, we don't actively monitor the github mirror for pull requests. Please follow the steps described here to submit your patches. http://wiki.apache.org/hadoop/ZooKeeper/HowToContribute Thanks! --Michi
          Hide
          Chris Nauroth added a comment -

          I'm curious if there has been any resolution to this discussion. If there is a decision to maintain the packaging targets, then would someone please review ZOOKEEPER-2124? This would fix the rpm packaging for -alpha releases. (The hyphen character is illegal in an rpm version.) There were also some earlier rpm fixes discussed in an email thread that would need to be committed and credited to the original author.

          http://mail-archives.apache.org/mod_mbox/zookeeper-user/201212.mbox/%3C50D2D481.8010507@pt-consulting.eu%3E

          Show
          Chris Nauroth added a comment - I'm curious if there has been any resolution to this discussion. If there is a decision to maintain the packaging targets, then would someone please review ZOOKEEPER-2124 ? This would fix the rpm packaging for -alpha releases. (The hyphen character is illegal in an rpm version.) There were also some earlier rpm fixes discussed in an email thread that would need to be committed and credited to the original author. http://mail-archives.apache.org/mod_mbox/zookeeper-user/201212.mbox/%3C50D2D481.8010507@pt-consulting.eu%3E
          Hide
          Flavio Junqueira added a comment -

          I don't think there is a resolution. Ralph Tice isn't in favor of relying only on BigTop, but frankly I don't understand if there is a strong reason behind it. If we are to maintain our own packaging, then I'd like to know how is interested in maintaining. In any case, I'll have a look at ZOOKEEPER-2124.

          Show
          Flavio Junqueira added a comment - I don't think there is a resolution. Ralph Tice isn't in favor of relying only on BigTop, but frankly I don't understand if there is a strong reason behind it. If we are to maintain our own packaging, then I'd like to know how is interested in maintaining. In any case, I'll have a look at ZOOKEEPER-2124 .
          Hide
          Ralph Tice added a comment -

          There definitely wasn't a strong reason, and I've come around on this issue. I don't think I'm supposed to minus or plus things either, sorry.

          Show
          Ralph Tice added a comment - There definitely wasn't a strong reason, and I've come around on this issue. I don't think I'm supposed to minus or plus things either, sorry.
          Hide
          Patrick Hunt added a comment -

          I'd say the main issues were/are that the original packaging was added (prior to bigtop existing btw), it worked ok, but there were bugs. Maintaining the different package types, ensuring they work across all the distros, versions, etc... is not a simple job. Not many/any folks in this community really understand all the packaging issues. Bigtop does and their charter is to create packaging. So it would make more sense imo to handle packaging there. At this point the packaging is out of date, there are more issues and they haven't been resolved. My thought is that it would make more sense to spend the effort once, at Bigtop, rather than twice, once at Bigtop and once here.

          Show
          Patrick Hunt added a comment - I'd say the main issues were/are that the original packaging was added (prior to bigtop existing btw), it worked ok, but there were bugs. Maintaining the different package types, ensuring they work across all the distros, versions, etc... is not a simple job. Not many/any folks in this community really understand all the packaging issues. Bigtop does and their charter is to create packaging. So it would make more sense imo to handle packaging there. At this point the packaging is out of date, there are more issues and they haven't been resolved. My thought is that it would make more sense to spend the effort once, at Bigtop, rather than twice, once at Bigtop and once here.
          Hide
          Chris Nauroth added a comment -

          Patrick Hunt, that makes a lot of sense. Speaking for myself, I can fumble through some packaging work, but I can't claim expertise in all of the variations that you mentioned.

          In Hadoop, we've made the decision to publish releases in tarball form only and defer all other packaging concerns to Bigtop. One possible additional consideration for ZooKeeper is that some users deploy it completely outside of the context of Hadoop. My own first use cases involved coordinating a distributed cache update and putting a distributed lock around some Cassandra schema modification operations that were not safe for concurrent execution at the time. Nothing else from the Bigtop ecosystem was relevant in this deployment. For those kinds of users, perhaps it's valuable for ZooKeeper to have its own release cadence with packaging support, decoupled from the Bigtop release cadence. It's definitely a trade-off though because of the maintenance costs.

          I'm attaching a patch that removes the ant rpm and deb targets and all supporting properties and files. Between this and ZOOKEEPER-2124, I think one will need to get committed and the other will be a no-fix. Alternatively, if this one goes in on trunk, then maybe ZOOKEEPER-2124 still goes into older maintenance release branches to minimize surprises for users on those branches.

          Thanks!

          Show
          Chris Nauroth added a comment - Patrick Hunt , that makes a lot of sense. Speaking for myself, I can fumble through some packaging work, but I can't claim expertise in all of the variations that you mentioned. In Hadoop, we've made the decision to publish releases in tarball form only and defer all other packaging concerns to Bigtop. One possible additional consideration for ZooKeeper is that some users deploy it completely outside of the context of Hadoop. My own first use cases involved coordinating a distributed cache update and putting a distributed lock around some Cassandra schema modification operations that were not safe for concurrent execution at the time. Nothing else from the Bigtop ecosystem was relevant in this deployment. For those kinds of users, perhaps it's valuable for ZooKeeper to have its own release cadence with packaging support, decoupled from the Bigtop release cadence. It's definitely a trade-off though because of the maintenance costs. I'm attaching a patch that removes the ant rpm and deb targets and all supporting properties and files. Between this and ZOOKEEPER-2124 , I think one will need to get committed and the other will be a no-fix. Alternatively, if this one goes in on trunk, then maybe ZOOKEEPER-2124 still goes into older maintenance release branches to minimize surprises for users on those branches. Thanks!
          Hide
          Patrick Hunt added a comment -

          If you're rolling your own (not using Bigtop packaging) they you typically start from tarball anyway. Even our scripts are of limited use in most of those cases. What I've seen is folks typically start from class files (jar) and work their way up from that for their particular context.

          Show
          Patrick Hunt added a comment - If you're rolling your own (not using Bigtop packaging) they you typically start from tarball anyway. Even our scripts are of limited use in most of those cases. What I've seen is folks typically start from class files (jar) and work their way up from that for their particular context.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12726926/ZOOKEEPER-1604.001.patch
          against trunk revision 1672934.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2643//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2643//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2643//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726926/ZOOKEEPER-1604.001.patch against trunk revision 1672934. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2643//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2643//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2643//console This message is automatically generated.
          Hide
          Chris Nauroth added a comment -

          -1 tests included. The patch doesn't appear to include any new or modified tests.

          There are no new tests, because this is a change in the build only. My manual test procedure was:

          1. Verify ant rpm does nothing/fails.
          2. Verify ant deb does nothing/fails.
          3. Verify ant package-native tar succeeds. Deploy the resulting zookeeper-<VERSION>.tar.gz and zookeeper-<VERSION>-lib.tar.gz. Run a single node and exercise some basic create/get/set commands.
          Show
          Chris Nauroth added a comment - -1 tests included. The patch doesn't appear to include any new or modified tests. There are no new tests, because this is a change in the build only. My manual test procedure was: Verify ant rpm does nothing/fails. Verify ant deb does nothing/fails. Verify ant package-native tar succeeds. Deploy the resulting zookeeper-<VERSION>.tar.gz and zookeeper-<VERSION>-lib.tar.gz. Run a single node and exercise some basic create/get/set commands.
          Hide
          Michi Mutsuzaki added a comment -

          How does Bigtop package ZooKeeper? It's not a standalone rpm/deb package?

          Show
          Michi Mutsuzaki added a comment - How does Bigtop package ZooKeeper? It's not a standalone rpm/deb package?
          Hide
          Chris Nauroth added a comment -

          The Bigtop rpm/deb packages are stand-alone. Those are visible here:

          http://bigtop.s3.amazonaws.com/

          It's definitely possible to install just a Zookeeper package from Bigtop. The point was more about discussing whether or not it would be valuable to offer rpm/deb packages in ZooKeeper's own release cycle, independent of Bigtop. As Patrick points out, there is always the tarball if a user needs to deploy a patch urgently before Bigtop publishes a package.

          Show
          Chris Nauroth added a comment - The Bigtop rpm/deb packages are stand-alone. Those are visible here: http://bigtop.s3.amazonaws.com/ It's definitely possible to install just a Zookeeper package from Bigtop. The point was more about discussing whether or not it would be valuable to offer rpm/deb packages in ZooKeeper's own release cycle, independent of Bigtop. As Patrick points out, there is always the tarball if a user needs to deploy a patch urgently before Bigtop publishes a package.

            People

            • Assignee:
              Patrick Hunt
              Reporter:
              Patrick Hunt
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:

                Development