[BIGTOP-2764] deployment failure when roles include spark::common and spark::yarn*

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.2.1
• Component/s: deployment
• Assignee: Kevin Monroe
• Reporter: Kevin Monroe
• Created: 10/May/17 15:24
• Updated: 16/May/17 15:07
• Resolved: 12/May/17 17:01

Description

I use bigtop roles to control what gets deployed. When I include `spark-client` and `spark-on-yarn`, I get the following deployment failure:

unit-spark-0: 15:15:24 INFO unit.spark/0.config-changed Error: Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: Package[spark-datanucleus] is already declared in file /home/ubuntu/bigtop.release/bigtop-1.2.0/bigtop-deploy/puppet/modules/spark/manifests/init.pp:158; cannot redeclare at /home/ubuntu/bigtop.release/bigtop-1.2.0/bigtop-deploy/puppet/modules/spark/manifests/init.pp:132 at /home/ubuntu/bigtop.release/bigtop-1.2.0/bigtop-deploy/puppet/modules/spark/manifests/init.pp:132:5 on node machine-0.qnvvwf5bfm5uzgvdpofqo5megg.dx.internal.cloudapp.net

This is because the spark-datanucleus package is defined for both the common class:

https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/spark/manifests/init.pp#L158

and the yarn classes:

https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/spark/manifests/init.pp#L117

Issue Links

• relates to: BIGTOP-2268 Untangle the convoluted Hive-support from Spark deployment
• links to: GitHub Pull Request #205