Details
-
Bug
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
1.1.0
-
None
-
None
Description
When deploying a standalone resourcemanager, puppet apply fails because hadoop::init_hdfs is not included in the resourcemanager role. This isn't a problem when deploying both a namenode and resource manager on the same unit (as is typically done on the head node), but it is problematic when you try to deploy RM by itself with no hdfs components.
Reproduce with the following in ./bigtop-deploy/puppet/hieradata/site.yaml:
'bigtop::roles_enabled': true
'bigtop::roles': resourcemanager
Then run puppet apply from ./bigtop-1.1.0:
puppet apply -d --modulepath=bigtop-deploy/puppet/modules:/etc/puppet/modules bigtop-deploy/puppet/manifests/site.pp
...
Error: Could not find resource 'Class[Hadoop::Init_hdfs]' for relationship on 'Class[Hadoop::Resourcemanager]' on node <xyz>
...
This is fixable in a couple different ways. First, just "include hadoop::init_hdfs" in the resourcemanager role in init.pp. That feels a bit heavy handed as I don't think the RM really needs to handle "init_hdfs".
The second solution is to remove the two Init_hdfs references from the resourcemanger role. This feels safe to me, and is what we're using in the bigtop juju charms to deploy a standalone resourcemanager.
I'll attach a patch shortly, but I'm interested in thoughts about this approach. Presumably, if you're deploying a standalone RM, you've got a namenode somewhere and you've already done the init_hdfs steps. Are there any scenarios where we really need the RM to do this?
I was thinking about the history server storing info in hdfs, so perhaps it could be a problem if somehow you tried to start RM with a historyserver hdfs location before hdfs was available... but I dont know why someone would do that.