Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2191

openjdk-7 is missing in bigtop/slave:ubuntu-14.04-ppc64le docker image

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.0.1, 1.1.0
    • Component/s: build, docker, ppc64le, toolchain
    • Labels:
      None
    • Environment:

      Ubuntu 14.04 on ppc64le

      Description

      openjdk-7-jdk is missing due to openjdk install failure.
      I believe the failure was caused by previous failure of "apt-get update".
      I have a strong suspicion the root of the problem resides in
      the way puppetlabs was installed during bigtop/puppet:ubuntu-14.04-ppc64le image build process.
      Debugging....

      1. BIGTOP-2191
        1 kB
        Amir Sanjar
      2. BIGTOP-2191.2.patch
        1 kB
        Olaf Flebbe

        Issue Links

          Activity

          Hide
          asanjar Amir Sanjar added a comment -

          initial debug result:
          "apt-get update" fails due to invalid platform architecture of puppetlabs-release package.
          It is tagged as powerpc and not ppc64el ??!!!!!! Verified the finding by
          https://apt.puppetlabs.com/dists/trusty/Release

          This is a regression caused by BIGTOP-2003.
          A possible solution could be to block "apt-get update" for puppetlabs package .

          Show
          asanjar Amir Sanjar added a comment - initial debug result: "apt-get update" fails due to invalid platform architecture of puppetlabs-release package. It is tagged as powerpc and not ppc64el ??!!!!!! Verified the finding by https://apt.puppetlabs.com/dists/trusty/Release This is a regression caused by BIGTOP-2003 . A possible solution could be to block "apt-get update" for puppetlabs package .
          Hide
          asanjar Amir Sanjar added a comment -

          Adding patch for review

          Show
          asanjar Amir Sanjar added a comment - Adding patch for review
          Hide
          asanjar Amir Sanjar added a comment -

          after applying this patch, we need to rebuild bigtop/puppet:ubuntu-14.04-ppc64le and bigtop/slave:ubuntu-14.04-ppc64le docker images

          Show
          asanjar Amir Sanjar added a comment - after applying this patch, we need to rebuild bigtop/puppet:ubuntu-14.04-ppc64le and bigtop/slave:ubuntu-14.04-ppc64le docker images
          Hide
          oflebbe Olaf Flebbe added a comment -

          The bigtop-puppet images do not contain java. The bigtop/slave images should do. If not, there is something wrong, surely.

          If you remove the file from sources.list.d you are reverting the installation process of the puppetlabs debian package. We need this because the stock ubuntu puppet packages are outdated, hiera is broken in it.

          I am busy with other things, but tonight I managed to upload the bigtop/puppet:ubuntu-14.04.ppc64le image so your patch to bigtop-slaves should work now ... proceesing bigtop/slave:ubuntu-14.04-ppc64le tomorrow.

          Show
          oflebbe Olaf Flebbe added a comment - The bigtop-puppet images do not contain java. The bigtop/slave images should do. If not, there is something wrong, surely. If you remove the file from sources.list.d you are reverting the installation process of the puppetlabs debian package. We need this because the stock ubuntu puppet packages are outdated, hiera is broken in it. I am busy with other things, but tonight I managed to upload the bigtop/puppet:ubuntu-14.04.ppc64le image so your patch to bigtop-slaves should work now ... proceesing bigtop/slave:ubuntu-14.04-ppc64le tomorrow.
          Hide
          asanjar Amir Sanjar added a comment -

          Hi Olaf,
          I do understand that java should get installed as part of bigtop/salves images creation. However, openjdk installation fails as shown by appended error message below. I tracked the origin of the failure to puppetlabs installation during bigtop/puppet image creation. As I explained above, the puppetlabs-release has been packaged with "powerpc" and not "ppc64le" as the architecture. As a result, all subsequent "apt-get update" fail. After applying the above patch I have been able to successfully build hadoop, spark, and Hue with:
          docker run -v `pwd`:/ws bigtop/slaves:ubuntu-14.04-ppc64le bash -l -c 'cd /ws ; ./gradlew spark-deb'

          /Stage[main]/Bigtop_toolchain::Jdk/Package[openjdk-7-jdk]: Dependency Exec[apt-update] has failures: true
          Warning: /Stage[main]/Bigtop_toolchain::Jdk/Package[openjdk-7-jdk]: Skipping because of failed dependencies

          Show
          asanjar Amir Sanjar added a comment - Hi Olaf, I do understand that java should get installed as part of bigtop/salves images creation. However, openjdk installation fails as shown by appended error message below. I tracked the origin of the failure to puppetlabs installation during bigtop/puppet image creation. As I explained above, the puppetlabs-release has been packaged with "powerpc" and not "ppc64le" as the architecture. As a result, all subsequent "apt-get update" fail. After applying the above patch I have been able to successfully build hadoop, spark, and Hue with: docker run -v `pwd`:/ws bigtop/slaves:ubuntu-14.04-ppc64le bash -l -c 'cd /ws ; ./gradlew spark-deb' /Stage [main] /Bigtop_toolchain::Jdk/Package [openjdk-7-jdk] : Dependency Exec [apt-update] has failures: true Warning: /Stage [main] /Bigtop_toolchain::Jdk/Package [openjdk-7-jdk] : Skipping because of failed dependencies
          Hide
          oflebbe Olaf Flebbe added a comment - - edited

          ah ok, I do understand now:

          The puppetlabs apt repository which is installed by the package https://apt.puppetlabs.com/puppetlabs-release-trusty.deb does only provide binaries for ppc64le so we cannot add it here. Otherwise we get errors when doing apt-get update.

          Then main reason we have added this repository is a workaround for BIGTOP-2003, because the ruby-hiera package at version 1.3.0 will trigger problems at deploy time.

          I conclude we either cannot build packages or cannot deploy them on ubuntu-14.04 on powerpc 64bit little endian.

          I propose to switch to ubuntu 15.04 which has ruby-hiera at 1.4 level AFAIK and drop ubuntu 14.04

          Show
          oflebbe Olaf Flebbe added a comment - - edited ah ok, I do understand now: The puppetlabs apt repository which is installed by the package https://apt.puppetlabs.com/puppetlabs-release-trusty.deb does only provide binaries for ppc64le so we cannot add it here. Otherwise we get errors when doing apt-get update . Then main reason we have added this repository is a workaround for BIGTOP-2003 , because the ruby-hiera package at version 1.3.0 will trigger problems at deploy time. I conclude we either cannot build packages or cannot deploy them on ubuntu-14.04 on powerpc 64bit little endian. I propose to switch to ubuntu 15.04 which has ruby-hiera at 1.4 level AFAIK and drop ubuntu 14.04
          Hide
          oflebbe Olaf Flebbe added a comment -

          My proposal for a workaround (but beware, deploy will break!)

          Show
          oflebbe Olaf Flebbe added a comment - My proposal for a workaround (but beware, deploy will break!)
          Hide
          asanjar Amir Sanjar added a comment - - edited

          +1 for your proposed workaround.
          and +1 on 15.04 support for the near future, I

          Show
          asanjar Amir Sanjar added a comment - - edited +1 for your proposed workaround. and +1 on 15.04 support for the near future, I

            People

            • Assignee:
              asanjar Amir Sanjar
              Reporter:
              asanjar Amir Sanjar
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:
                Resolved:

                Development