[WHIRR-733] Need to support a new option called whirr.user-thread when it comes to the Openstack

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.8.2
• Fix Version/s: 0.9.0
• Component/s: core

Environment

Ubuntu
Openstack

• Assignee: Unassigned
• Reporter: Jeremy Lv
• Created: 26/Jun/13 13:37
• Updated: 05/Jul/13 03:25

Description

I found it will be blocked when create a virtual machine based on the Openstack(It is because the whirr project can't remove the swift container after created the new instance)

Here's my reproduced steps:

First of all, you must configrate the Openstack environments including the openstack swift

1>. Configurate my properties file called zookeeper.properties(In my environment, I have put the file in the directory of "/home/Jeremy/zookeeper.properties")

whirr.cluster-name=myhadoopcluster
#whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1 hadoop-datanode+hadoop-tasktracker
whirr.instance-templates=1 zookeeper

whirr.provider=openstack-nova
whirr.location-id=RegionOne
whirr.identity=admin:admin
whirr.credential=hastexo
whirr.image-id=RegionOne/7cf21d93-d04a-4fb4-a09b-f546a9461726
whirr.endpoint = http://10.167.157.86:5000/v2.0
whirr.blobstore-provider=swift-keystone
whirr.blobstore-endpoint=http://10.167.157.86:5000/v2.0/
whirr.blobstore-identity=service:swift
whirr.blobstore-credential=hastexo
#whirr.blobstore-cache-container=swift
#whirr.blobstore-location-id=sdb1
whirr.zookeeper.tarball.url=file:///home/Jeremy/test_sample1.war
jclouds.openstack-nova.auto-generate-keypairs = true
#jclouds.openstack-nova.auto-create-floating-ips = true

whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
whirr.public-key-file=${sys:user.home}/.ssh/id_rsa.pub

2>. Download the whirr-0.8.2.zip to my local platform(/home/Jeremy/whirr-0.8.2) and unzip it

3>. cd /home/Jeremy/whirr-0.8.2 and execute the following command:
./bin/whirr launch-cluster --config ../zookeeper.properties

4>.You will found the swift blob container can be created successfully but can't be removed automatically after create the new instance.(It will be blocked during the process of launch-cluster and can't be canced).

After a further investigation about the whirr project, I found it is because of the following code:

BlobCache.java

  static {
    /* Ensure that all created containers are removed when the JVM stops */
    Runtime.getRuntime().addShutdownHook(new Thread() {
      @Override
      public void run() {
        BlobCache.dropAndCloseAll();
      }
    });
  }

It is because the default value of the user thread in jclouds side is "0", even we create a new thread in the whirr side, we can't delete the container here because of the following Exception

Exception in thread "Thread-0" java.util.concurrent.RejectedExecutionException: Task [delegate=[method=SwiftKeystoneAsyncClient.listContainers, request=GET http://10.167.157.86:8080/v1/AUTH_60d91f3ee474439e8edbbf9f15f424c1/?format=json HTTP/1.1], executionList=com.google.common.util.concurrent.ExecutionList@1b8633e] rejected from java.util.concurrent.ThreadPoolExecutor@7b7e61[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8]
	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2013)
	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1337)
	at org.jclouds.concurrent.config.DescribingExecutorService.execute(DescribingExecutorService.java:105)
	at org.jclouds.concurrent.Futures$FutureListener.addListener(Futures.java:122)
	at org.jclouds.concurrent.Futures$ListenableFutureAdapter.addListener(Futures.java:153)
	at com.google.common.util.concurrent.Futures.transform(Futures.java:269)
	at com.google.common.util.concurrent.Futures.transform(Futures.java:380)
	at org.jclouds.concurrent.Futures.compose(Futures.java:270)
	at org.jclouds.http.TransformingHttpCommandExecutorServiceImpl.submit(TransformingHttpCommandExecutorServiceImpl.java:54)
	at org.jclouds.http.TransformingHttpCommandImpl.execute(TransformingHttpCommandImpl.java:73)
	at org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFutureForHttpRequestMappedToMethodAndArgs(AsyncRestClientProxy.java:254)
	at org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:148)
	at $Proxy70.listContainers(Unknown Source)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:170)
	at $Proxy71.listContainers(Unknown Source)
	at com.jclouds.test.JCloudsSwift.deleteConatiner(JCloudsSwift.java:63)
	at com.jclouds.test.JCloudsSwift.access$0(JCloudsSwift.java:60)
	at com.jclouds.test.JCloudsSwift$1.run(JCloudsSwift.java:34)

All in all, I have also write a sample related to the jcloud and found if we define the PROPERTY_USER_THREADS in the jclouds side, it will be successful to remove the swift container though the Runtime.getRuntime().addShutdownHook(new Thread(). If we didn't define the PROPERTY_USER_THREADS in the jclouds side and it will be thrown the RejectedExecutionException in calling the Blobstore.deleteContainer method because of the default user thread is "0"

So I think we need to define a new option to support this situation.