Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2796

Bigtop Zookeeper(3.4.6) package conflict with Ubuntu 16.04 Zookeeper(3.4.8)

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.2.1
    • Component/s: None
    • Labels:
      None

      Description

      qThis is discovered by my recently work to produce 1.2.1 sandboxes:

      https://ci.bigtop.apache.org/view/Docker/job/Docker-Sandbox/

      Specifically, only HBase sandbox on ubuntu 16.04 is failing:
      https://ci.bigtop.apache.org/view/Docker/job/Docker-Sandbox/BUILD_ENVIRONMENTS=ubuntu-16.04,STACK=hdfs_hbase,label=docker-slave-06/51/console

      The log:

      Error: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install zookeeper-server' returned 100: Reading package lists...
      Building dependency tree...
      Reading state information...
      Some packages could not be installed. This may mean that you have
      requested an impossible situation or if you are using the unstable
      distribution that some required packages have not yet been created
      or been moved out of Incoming.
      The following information may help to resolve the situation:
      
      The following packages have unmet dependencies:
       zookeeper-server : Depends: zookeeper (= 3.4.6-1) but 3.4.8-1 is to be installed
      E: Unable to correct problems, you have held broken packages.
      Error: /Stage[main]/Hadoop_zookeeper::Server/Package[zookeeper-server]/ensure: change from purged to latest failed: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install zookeeper-server' returned 100: Reading package lists...
      Building dependency tree...
      Reading state information...
      Some packages could not be installed. This may mean that you have
      requested an impossible situation or if you are using the unstable
      distribution that some required packages have not yet been created
      or been moved out of Incoming.
      The following information may help to resolve the situation:
      
      The following packages have unmet dependencies:
       zookeeper-server : Depends: zookeeper (= 3.4.6-1) but 3.4.8-1 is to be installed
      E: Unable to correct problems, you have held broken packages.
      

      The same problem can be found in Provisioner as well:
      https://ci.bigtop.apache.org/view/Provisioner/job/Bigtop-trunk-deployments/BUILD_SLAVE=docker-slave-02,COMPONENTS=hbase,OS=ubuntu_xenial/43/console

        Issue Links

          Activity

          Hide
          oflebbe Olaf Flebbe added a comment -

          Hi Evans Ye : Hope you are planning to use apt::pin with puppet to resolve this?

          Show
          oflebbe Olaf Flebbe added a comment - Hi Evans Ye : Hope you are planning to use apt::pin with puppet to resolve this?
          Hide
          kwmonroe Kevin W Monroe added a comment -

          +1 to Olaf's pin suggestion.. We could pin the zookeeper package by itself, but I prefer pinning the Bigtop repo here:

          https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/manifests/site.pp#L57

          500 is the default pin priority for ubuntu repositories, so I suggest adding a "pin => 900" attribute to the Bigtop apt::source above.

          This will force apt to look in the Bigtop repos first (900 > 500) for any package operations.

          One downside of this is that it only benefits people that use puppet to deploy bigtop. For people that manually add the bigtop repo to an ubuntu 16.04 system, they'll need to do their own pinning to ensure packages come from bigtop first.

          Show
          kwmonroe Kevin W Monroe added a comment - +1 to Olaf's pin suggestion.. We could pin the zookeeper package by itself, but I prefer pinning the Bigtop repo here: https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/manifests/site.pp#L57 500 is the default pin priority for ubuntu repositories, so I suggest adding a "pin => 900" attribute to the Bigtop apt::source above. This will force apt to look in the Bigtop repos first (900 > 500) for any package operations. One downside of this is that it only benefits people that use puppet to deploy bigtop. For people that manually add the bigtop repo to an ubuntu 16.04 system, they'll need to do their own pinning to ensure packages come from bigtop first.
          Hide
          rvs Roman Shaposhnik added a comment -

          +1 to pinning! Also, whoever writes the actual code – please add a comment – 'cuz this one comes up every so oftern by I always forget how to deal with it

          Show
          rvs Roman Shaposhnik added a comment - +1 to pinning! Also, whoever writes the actual code – please add a comment – 'cuz this one comes up every so oftern by I always forget how to deal with it
          Hide
          evans_ye Evans Ye added a comment -

          Patch attached. I've tested this with Zookeeper and HBase components using Provisioner.
          Note that this patch requires BIGTOP-2775 first.

          Show
          evans_ye Evans Ye added a comment - Patch attached. I've tested this with Zookeeper and HBase components using Provisioner. Note that this patch requires BIGTOP-2775 first.
          Hide
          oflebbe Olaf Flebbe added a comment -

          Thanks Evans Ye, patch is surprisingly easy. Works for me as well. will commit it in a minute.

          Show
          oflebbe Olaf Flebbe added a comment - Thanks Evans Ye , patch is surprisingly easy. Works for me as well. will commit it in a minute.
          Hide
          oflebbe Olaf Flebbe added a comment -

          Thanks!

          Show
          oflebbe Olaf Flebbe added a comment - Thanks!

            People

            • Assignee:
              evans_ye Evans Ye
              Reporter:
              evans_ye Evans Ye
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development