Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2616

refresh juju hadoop-processing bundle

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.2.0
    • Component/s: deployment
    • Labels:
      None

      Description

      We've made a number of improvements to the hadoop-processing bundle that are ready for review and inclusion upstream:

      • ganglia/rsyslog have been updated from trusty to xenial
      • add new machine constraints since aws now defaults to an 8gb root disk (we want 32g at a minimum)
      • ensure applications are actually ready before running tests
      • ensure the proper amulet test harness is included in the bundle tests

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user kwmonroe opened a pull request:

          https://github.com/apache/bigtop/pull/167

          BIGTOP-2616: refresh juju hadoop-processing bundle

          We've made a number of improvements to the hadoop-processing bundle that are ready for review and inclusion upstream:

          • ganglia/rsyslog have been updated from trusty to xenial
          • add new machine constraints since aws now defaults to an 8gb root disk (we want 32g at a minimum)
          • ensure applications are actually ready before running tests
          • ensure the proper amulet test harness is included in the bundle tests

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/juju-solutions/bigtop bug/hp-cwr-fixes

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/bigtop/pull/167.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #167


          commit e343d68c453a17b1be60b564ce075664bd902f6b
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2016-10-27T20:10:13Z

          bump charm revnos and update bundle test to address CWR failures; this also addresses an amulet issue where the bundle test would not execute unless we removed to: charm placement directives

          commit fabda963a74d2e746d10c9503ba7286f256aef10
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2016-10-28T15:45:27Z

          wait for all apps to report ready in amulet test

          commit 013c88f234ddfd8fed2dbd97fd704a36d6f087dc
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2016-10-28T16:46:11Z

          slave tests are barely finishing, so up action timeout to 1hr

          commit 7ae03c2d016f1ded2bc2b49609fa4f001576bc0c
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2016-10-28T16:59:32Z

          fix amulet test comment

          commit 0173e04102a71629377d992ef4edefd35ef92d09
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2016-10-28T23:13:31Z

          add working ganglia unit

          commit 71a5d39165870d0781f9f2f0e578b2cb1f7b632b
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2016-10-31T16:37:31Z

          skip slave smoke tests and update smoke failure message

          commit 143cc6c3bbddf856a0627a06ed219908eb60985f
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2016-11-08T15:59:27Z

          bump ganglia revnos to pick up app version and install hook failures

          commit bc31b83e0013671b1d4af3a7b666cb970958ed57
          Author: Cory Johns <johnsca@gmail.com>
          Date: 2016-11-14T21:49:53Z

          hadoop-processing tests.yaml should include PPA for Amulet

          commit c9ca70414447021eb249464cb5e06f4c91a1fa8f
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2016-11-15T18:36:02Z

          bump rsyslog charm revs for lint fix

          commit d9b5a90a0b3d66ed7561471b3abee28d8ee7fc06
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2016-11-23T01:41:35Z

          charm rev bump, new constraints

          • bump core hadoop charms to latest (new swap handler, detect failed service
            start, add spark user/group to nn/rm)
          • aws defaults to 8gb root filesystem; bump this to 20gb

          commit 8888e2baa868afda684785fd3d6cfb4f1495c301
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2016-11-28T23:10:01Z

          workaround azure root-disk: https://bugs.launchpad.net/juju/+bug/1645408


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/167 BIGTOP-2616 : refresh juju hadoop-processing bundle We've made a number of improvements to the hadoop-processing bundle that are ready for review and inclusion upstream: ganglia/rsyslog have been updated from trusty to xenial add new machine constraints since aws now defaults to an 8gb root disk (we want 32g at a minimum) ensure applications are actually ready before running tests ensure the proper amulet test harness is included in the bundle tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/juju-solutions/bigtop bug/hp-cwr-fixes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/bigtop/pull/167.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #167 commit e343d68c453a17b1be60b564ce075664bd902f6b Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2016-10-27T20:10:13Z bump charm revnos and update bundle test to address CWR failures; this also addresses an amulet issue where the bundle test would not execute unless we removed to: charm placement directives commit fabda963a74d2e746d10c9503ba7286f256aef10 Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2016-10-28T15:45:27Z wait for all apps to report ready in amulet test commit 013c88f234ddfd8fed2dbd97fd704a36d6f087dc Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2016-10-28T16:46:11Z slave tests are barely finishing, so up action timeout to 1hr commit 7ae03c2d016f1ded2bc2b49609fa4f001576bc0c Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2016-10-28T16:59:32Z fix amulet test comment commit 0173e04102a71629377d992ef4edefd35ef92d09 Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2016-10-28T23:13:31Z add working ganglia unit commit 71a5d39165870d0781f9f2f0e578b2cb1f7b632b Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2016-10-31T16:37:31Z skip slave smoke tests and update smoke failure message commit 143cc6c3bbddf856a0627a06ed219908eb60985f Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2016-11-08T15:59:27Z bump ganglia revnos to pick up app version and install hook failures commit bc31b83e0013671b1d4af3a7b666cb970958ed57 Author: Cory Johns <johnsca@gmail.com> Date: 2016-11-14T21:49:53Z hadoop-processing tests.yaml should include PPA for Amulet commit c9ca70414447021eb249464cb5e06f4c91a1fa8f Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2016-11-15T18:36:02Z bump rsyslog charm revs for lint fix commit d9b5a90a0b3d66ed7561471b3abee28d8ee7fc06 Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2016-11-23T01:41:35Z charm rev bump, new constraints bump core hadoop charms to latest (new swap handler, detect failed service start, add spark user/group to nn/rm) aws defaults to 8gb root filesystem; bump this to 20gb commit 8888e2baa868afda684785fd3d6cfb4f1495c301 Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2016-11-28T23:10:01Z workaround azure root-disk: https://bugs.launchpad.net/juju/+bug/1645408
          Hide
          kwmonroe Kevin W Monroe added a comment -

          Bundle is available at:

          https://jujucharms.com/hadoop-processing/

          Daily tests of this bundle across various clouds are available here:

          http://data.vapour.ws/cwr-tests/results/bundle_hadoop_processing/index.html

          Show
          kwmonroe Kevin W Monroe added a comment - Bundle is available at: https://jujucharms.com/hadoop-processing/ Daily tests of this bundle across various clouds are available here: http://data.vapour.ws/cwr-tests/results/bundle_hadoop_processing/index.html
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user johnsca commented on the issue:

          https://github.com/apache/bigtop/pull/167

          This looks good to me. The tests are more specific, and are all passing for deployments on AWS and Azure, as evinced by http://cwr.vapour.ws/bundle_hadoop_processing/aba491a0c58843e0ab60d7c5291bf135/report.html

          :+1:

          Show
          githubbot ASF GitHub Bot added a comment - Github user johnsca commented on the issue: https://github.com/apache/bigtop/pull/167 This looks good to me. The tests are more specific, and are all passing for deployments on AWS and Azure, as evinced by http://cwr.vapour.ws/bundle_hadoop_processing/aba491a0c58843e0ab60d7c5291bf135/report.html :+1:
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/bigtop/pull/167

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/bigtop/pull/167

            People

            • Assignee:
              kwmonroe Kevin W Monroe
              Reporter:
              kwmonroe Kevin W Monroe
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development