[WHIRR-729] Support default value of whirr.blobstore-provider when it comes to the Openstack swift

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 0.8.2
• Fix Version/s: 0.9.0
• Component/s: core

Environment

Ubuntu,Openstack

• Assignee: Unassigned
• Reporter: Jeremy Lv
• Created: 21/Jun/13 11:58
• Updated: 24/Jun/13 03:17

Description

I have try to launch a cluster based on the environment of Openstack and found it will be failed when we didn't specify the value of whirr.blobstore-provider. Here's some of the failed informations:

2013-06-21 19:36:52,428 ERROR [org.apache.whirr.ClusterController] (main) Unable to start the cluster. Terminating all nodes.
com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: id must be defined
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2258)
	at com.google.common.cache.LocalCache.get(LocalCache.java:3990)
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3994)
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4878)
	at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4884)
	at org.apache.whirr.service.BlobStoreContextBuilder$Cache.apply(BlobStoreContextBuilder.java:72)
	at org.apache.whirr.service.BlobStoreContextBuilder.build(BlobStoreContextBuilder.java:101)
	at org.apache.whirr.util.BlobCache.<init>(BlobCache.java:95)
	at org.apache.whirr.util.BlobCache.getInstance(BlobCache.java:69)
	at org.apache.whirr.service.ClusterActionHandlerSupport.prepareRemoteFileUrl(ClusterActionHandlerSupport.java:239)
	at org.apache.whirr.service.zookeeper.ZooKeeperClusterActionHandler.beforeBootstrap(ZooKeeperClusterActionHandler.java:71)
	at org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:55)
	at org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:123)
	at org.apache.whirr.ClusterController.bootstrapCluster(ClusterController.java:137)
	at org.apache.whirr.ClusterController.launchCluster(ClusterController.java:113)
	at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:69)
	at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:59)
	at org.apache.whirr.cli.Main.run(Main.java:69)
	at org.apache.whirr.cli.Main.main(Main.java:102)
Caused by: java.lang.IllegalArgumentException: id must be defined
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92)
	at org.jclouds.util.Preconditions2.checkNotEmpty(Preconditions2.java:50)
	at org.jclouds.providers.ProviderPredicates.id(ProviderPredicates.java:107)
	at org.jclouds.providers.Providers.withId(Providers.java:103)
	at org.jclouds.ContextBuilder.newBuilder(ContextBuilder.java:164)
	at org.apache.whirr.service.BlobStoreContextBuilder$Cache$1.load(BlobStoreContextBuilder.java:83)
	at org.apache.whirr.service.BlobStoreContextBuilder$Cache$1.load(BlobStoreContextBuilder.java:78)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3589)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2374)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2337)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2252)
	... 18 more

Here's my property file:

whirr.cluster-name=myhadoopcluster
#whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1 hadoop-datanode+hadoop-tasktracker
whirr.instance-templates=1 zookeeper

whirr.provider=openstack-nova
whirr.location-id=RegionOne
whirr.identity=admin:admin
whirr.credential=hastexo
whirr.image-id=RegionOne/7cf21d93-d04a-4fb4-a09b-f546a9461726
whirr.endpoint = http://10.167.157.86:5000/v2.0
#whirr.blobstore-provider=swift-keystone
whirr.blobstore-endpoint=http://10.167.157.86:5000/v2.0/
whirr.blobstore-identity=service:swift
whirr.blobstore-credential=hastexo
#whirr.blobstore-cache-container=swift
#whirr.blobstore-location-id=swift-keystone
whirr.zookeeper.tarball.url=file:///home/Jeremy/test_sample1.war
jclouds.openstack-nova.auto-generate-keypairs = true
#jclouds.openstack-nova.auto-create-floating-ips = true

whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
whirr.public-key-file=${sys:user.home}/.ssh/id_rsa.pub

After detailed investigation about the whirr project, I found it is becasue the whirr-core component didn't specify the default value which will Computed from whirr.provider as documented in the offical specification:
http://whirr.apache.org/docs/0.8.2/configuration-guide.html

Here's my modification to support the specification which is documented in the official website

ClusterSpec.java

Index: ClusterSpec.java
===================================================================
--- ClusterSpec.java	(revision 1488828)
+++ ClusterSpec.java	(working copy)
@@ -576,6 +576,7 @@
     mappings.put("cloudservers", "cloudfiles-us");
     mappings.put("cloudservers-us", "cloudfiles-us");
     mappings.put("cloudservers-uk", "cloudfiles-uk");
+    mappings.put("openstack-nova", "swift-keystone");
 
     if (!mappings.containsKey(provider)) {
       return null;

Thanks

-Jeremy Lv