Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2764

deployment failure when roles include spark::common and spark::yarn*

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.2.1
    • Component/s: deployment
    • Labels:
      None

      Description

      I use bigtop roles to control what gets deployed. When I include `spark-client` and `spark-on-yarn`, I get the following deployment failure:

      unit-spark-0: 15:15:24 INFO unit.spark/0.config-changed Error: Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: Package[spark-datanucleus] is already declared in file /home/ubuntu/bigtop.release/bigtop-1.2.0/bigtop-deploy/puppet/modules/spark/manifests/init.pp:158; cannot redeclare at /home/ubuntu/bigtop.release/bigtop-1.2.0/bigtop-deploy/puppet/modules/spark/manifests/init.pp:132 at /home/ubuntu/bigtop.release/bigtop-1.2.0/bigtop-deploy/puppet/modules/spark/manifests/init.pp:132:5 on node machine-0.qnvvwf5bfm5uzgvdpofqo5megg.dx.internal.cloudapp.net
      

      This is because the spark-datanucleus package is defined for both the common class:

      https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/spark/manifests/init.pp#L158

      and the yarn classes:

      https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/spark/manifests/init.pp#L117

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                kwmonroe Kevin W Monroe
                Reporter:
                kwmonroe Kevin W Monroe
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: