Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-3123 Define Bigtop 1.5.0 release BOM
  3. BIGTOP-3343

Add Debian 10 and Ubuntu 18.04 support to the Docker provisioner

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.5.0
    • debian, docker, provisioner
    • None

    Description

      I tried to use bigtop/puppet:trunk-debian-10 and bigtop/puppet:trunk-ubuntu-18.04 with the Docker provisioner and found some problems:

      1. The systemd-sysv package has to be installed to ubuntu-18.04 in bigtop_toolchain/bin/puppetize.sh, otherwise the provisioner fails as follows:

      $ ./docker-hadoop.sh -c 1 -C config_ubuntu-18.04.yaml 
      
      (snip)
      
      Creating 20200421223222r15703_bigtop_1 ... error
      
      ERROR: for 20200421223222r15703_bigtop_1  Cannot start service bigtop: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/sbin/init\": stat /sbin/init: no such file or directory": unknown
      
      ERROR: for bigtop  Cannot start service bigtop: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/sbin/init\": stat /sbin/init: no such file or directory": unknown
      ERROR: Encountered errors while bringing up the project.
      

      2. APT::Get::AllowUnauthenticated 1; in bigtop-deploy/puppet/manifests/bigtop_repo.pp doesn't seem to work on these distros. It seems to be replaced with Acquire::AllowInsecureRepositories "true";.

      3. The gnupg package has to be installed to debian-10 in bigtop_toolchain/bin/puppetize.sh, otherwise the provisioner fails as follows:

      $ ./docker-hadoop.sh -c 1 -C config_debian-10.yaml 
      
      (snip)
      
      Error: Could not find a suitable provider for apt_key
      Notice: Applied catalog in 0.39 seconds
      
      [LOG] Failed to provision container a1eddad59e7f0ff1cbfd56d09ea9d78503676fbdff157ad85d95c6819b03e3fc with exit code 6
      

      4. The existing puppet manifest (bigtop-deploy/puppet/manifests/jdk.pp) tries to install openjdk-8-jdk as follows, but it's not provided by the distro on debian-10. Instead, we have to install adoptopenjdk 8 just like bigtop_toolchain/manifests/jdk.pp.

      $ ./docker-hadoop.sh -c 1 -C config_debian-10.yaml 
      
      (snip)
      
      Notice: /Stage[main]/Jdk/Apt::Pin[backports_jdk]/Apt::Setting[pref-backports_jdk]/File[/etc/apt/preferences.d/backports_jdk.pref]/ensure: defined content as '{md5}98444fb15be80221672742e79a046130'
      Notice: /Stage[main]/Jdk/Apt::Pin[backports_ca]/Apt::Setting[pref-backports_ca]/File[/etc/apt/preferences.d/backports_ca.pref]/ensure: defined content as '{md5}610a41d22559e3f19685788014d7d9df'
      Notice: /Stage[main]/Jdk/Exec[own_update]/returns: executed successfully
      Error: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install openjdk-8-jdk' returned 100: Reading package lists...
      Building dependency tree...
      Reading state information...
      E: Unable to locate package openjdk-8-jdk
      Error: /Stage[main]/Jdk/Package[jdk]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install openjdk-8-jdk' returned 100: Reading package lists...
      Building dependency tree...
      Reading state information...
      E: Unable to locate package openjdk-8-jdk
      Notice: /Stage[main]/Hadoop::Common/Package[hadoop]: Dependency Package[jdk] has failures: true
      

      Attachments

        Activity

          People

            sekikn Kengo Seki
            sekikn Kengo Seki
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h 10m
                2h 10m