Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.2.0
    • Component/s: deployment
    • Labels:
      None

      Description

      I refreshed charms in BIGTOP-2703. Our Juju CI system picked up some issues that should be addressed. None of these are show-stoppers, but all of them are issues that could affect the UX when deploying/testing bigtop charms:

      • Transient ganglia/rsyslog errors (e.g. websocket timeouts when trying to connect to http://ganglia-ip/ganglia). Since these are not directly related to bigtop, ignore test failures in these components.
      • hbase: this was neat. I just happened to run an hbase test and discovered the charm wasn't using the hbase-quorum interface to learn about hbase peers. Because of this, i'm adding a hadoop-hbase bundle that will surface issues in the HBase charm faster.
      • kafka, spark, and zookeeper: part of the tests for these charms alters the configuration. Ensure the charm-under-test does not actually alter the config of an existing deployment. This means deploy <charm>-test for testing, and destroy it when done.
      • mahout: the smoke test is hot garbage. Make it better (i.e. make it like the giraph smoke test).

      As mentioned earlier, none of these issues affect charm functionality, but they do resolve test issues that might bite a user. This should be a trivial add and I am confident I can get good test runs before 1.2 goes RC.

        Issue Links

          Activity

          Hide
          kwmonroe Kevin W Monroe added a comment -

          Roman Shaposhnik, if you'd give me like 5 minutes to write a comment, you'd know this is all buttoned up

          Show
          kwmonroe Kevin W Monroe added a comment - Roman Shaposhnik , if you'd give me like 5 minutes to write a comment, you'd know this is all buttoned up
          Hide
          kwmonroe Kevin W Monroe added a comment -

          I feel like this might be in the running for largest PR that doesn't actually affect functionality. We had numerous Juju CI issues that are fixed here, but I doubt any would be seen by real users. I'm confident in merging this for a few reasons:

          1) this only touches code related to juju (./bigtop-packages/src/charms and ./bigtop-deploy/juju)
          2) multiple recent "green" runs of Juju CI across multiple clouds:

          http://bigtop.charm.qa/cwr_bundle_hadoop_processing/24/report.html
          http://bigtop.charm.qa/cwr_bundle_spark_processing/22/report.html

          Show
          kwmonroe Kevin W Monroe added a comment - I feel like this might be in the running for largest PR that doesn't actually affect functionality. We had numerous Juju CI issues that are fixed here, but I doubt any would be seen by real users. I'm confident in merging this for a few reasons: 1) this only touches code related to juju (./bigtop-packages/src/charms and ./bigtop-deploy/juju) 2) multiple recent "green" runs of Juju CI across multiple clouds: http://bigtop.charm.qa/cwr_bundle_hadoop_processing/24/report.html http://bigtop.charm.qa/cwr_bundle_spark_processing/22/report.html
          Hide
          rvs Roman Shaposhnik added a comment -

          Kevin W Monroe I saw that the commit went in. Is there anything else left before we can close this issue?

          Show
          rvs Roman Shaposhnik added a comment - Kevin W Monroe I saw that the commit went in. Is there anything else left before we can close this issue?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/bigtop/pull/189

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/bigtop/pull/189
          Hide
          kwmonroe Kevin W Monroe added a comment -

          I'll provide links to final CI runs for these bits asap (these jobs are taking fooorever).

          Show
          kwmonroe Kevin W Monroe added a comment - I'll provide links to final CI runs for these bits asap (these jobs are taking fooorever).
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user kwmonroe opened a pull request:

          https://github.com/apache/bigtop/pull/189

          BIGTOP-2712: Juju CI driven updates

          The original point of this branch was to refresh the hbase charm (readme was out-of-date, needed lint love). However, as soon as ran CI on this and recent changes from #187, it was like I opened a filthy can of yarn (or however that saying goes).

          None of these are show-stoppers, but all of them are issues that could affect the UX when deploying/testing bigtop charms:

          • Transient ganglia/rsyslog errors (e.g. websocket timeouts when trying to connect to http://ganglia-ip/ganglia). Since these are not directly related to bigtop, ignore test failures in these components.
          • hbase: this was neat. I just happened to run an hbase test and discovered the charm wasn't using the hbase-quorum interface to learn about hbase peers. Because of this, i'm adding a hadoop-hbase bundle that will surface issues in the HBase charm faster.
          • kafka, spark, and zookeeper: part of the tests for these charms alters the configuration. Ensure the charm-under-test does not actually alter the config of an existing deployment. This means deploy `<charm>-test` for testing, and destroy it when done.
          • mahout: the smoke test is hot garbage. Make it better (i.e. make it like the giraph smoke test).

          As mentioned earlier, none of these issues affect charm functionality, but they do resolve test issues that might bite a user. This should be a trivial add and I am confident I can get good test runs before 1.2 goes RC.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/juju-solutions/bigtop feature/hbase-refresh

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/bigtop/pull/189.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #189


          commit b2d91c6cf164b5e46109a622eccb4c9f5e53b6d0
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2017-03-19T02:03:16Z

          readme updates (note actions and bundle deployments); lint fixes for actions; show app version in status output

          commit ac9ed8537abaecb46cf94eb7927cffd9fe736531
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2017-03-19T14:47:48Z

          adding hadoop-hbase bundle

          commit 039a61858cdfa754324efea22735d608eee04c9c
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2017-03-19T14:50:35Z

          wrong rev for hbase

          commit 3176339f61c048b4303aea1c172c42a7ead0c905
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2017-03-22T04:50:26Z

          Merge branch 'master' into feature/hbase-refresh

          commit da7e3cb845ad0135d1c251a236d62aa76bf8044b
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2017-03-22T05:38:10Z

          charm rev bump

          commit 63453c5d7ffbde8ec4da0bdbfdab00732424bff9
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2017-03-24T03:53:07Z

          charm updates from recent Juju CI

          • hbase should include quorum interface
          • kafka/mahout/spark/zk tests needed robustification

          commit 8152bd2eb174439123fc2fc83bcd7c93f48b15c7
          Author: Kevin W Monroe <kevin.monroe@canonical.com>
          Date: 2017-03-24T03:54:44Z

          juju bundle refresh to pick up latest charm revs and exclude tests for non-bigtop components


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/189 BIGTOP-2712 : Juju CI driven updates The original point of this branch was to refresh the hbase charm (readme was out-of-date, needed lint love). However, as soon as ran CI on this and recent changes from #187, it was like I opened a filthy can of yarn (or however that saying goes). None of these are show-stoppers, but all of them are issues that could affect the UX when deploying/testing bigtop charms: Transient ganglia/rsyslog errors (e.g. websocket timeouts when trying to connect to http://ganglia-ip/ganglia ). Since these are not directly related to bigtop, ignore test failures in these components. hbase: this was neat. I just happened to run an hbase test and discovered the charm wasn't using the hbase-quorum interface to learn about hbase peers. Because of this, i'm adding a hadoop-hbase bundle that will surface issues in the HBase charm faster. kafka, spark, and zookeeper: part of the tests for these charms alters the configuration. Ensure the charm-under-test does not actually alter the config of an existing deployment. This means deploy `<charm>-test` for testing, and destroy it when done. mahout: the smoke test is hot garbage. Make it better (i.e. make it like the giraph smoke test). As mentioned earlier, none of these issues affect charm functionality, but they do resolve test issues that might bite a user. This should be a trivial add and I am confident I can get good test runs before 1.2 goes RC. You can merge this pull request into a Git repository by running: $ git pull https://github.com/juju-solutions/bigtop feature/hbase-refresh Alternatively you can review and apply these changes as the patch at: https://github.com/apache/bigtop/pull/189.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #189 commit b2d91c6cf164b5e46109a622eccb4c9f5e53b6d0 Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2017-03-19T02:03:16Z readme updates (note actions and bundle deployments); lint fixes for actions; show app version in status output commit ac9ed8537abaecb46cf94eb7927cffd9fe736531 Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2017-03-19T14:47:48Z adding hadoop-hbase bundle commit 039a61858cdfa754324efea22735d608eee04c9c Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2017-03-19T14:50:35Z wrong rev for hbase commit 3176339f61c048b4303aea1c172c42a7ead0c905 Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2017-03-22T04:50:26Z Merge branch 'master' into feature/hbase-refresh commit da7e3cb845ad0135d1c251a236d62aa76bf8044b Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2017-03-22T05:38:10Z charm rev bump commit 63453c5d7ffbde8ec4da0bdbfdab00732424bff9 Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2017-03-24T03:53:07Z charm updates from recent Juju CI hbase should include quorum interface kafka/mahout/spark/zk tests needed robustification commit 8152bd2eb174439123fc2fc83bcd7c93f48b15c7 Author: Kevin W Monroe <kevin.monroe@canonical.com> Date: 2017-03-24T03:54:44Z juju bundle refresh to pick up latest charm revs and exclude tests for non-bigtop components

            People

            • Assignee:
              kwmonroe Kevin W Monroe
              Reporter:
              kwmonroe Kevin W Monroe
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Remaining Estimate - 24h
                24h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development