Whirr
  1. Whirr
  2. WHIRR-379

Provide a local cloud for testing using jclouds-virtualbox

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None

      Description

      Now that jclouds will have working virtualbox support in 1.5.X we should try and run local itests and have local recipes so that services can be experimented with without paying for mistakes.

      1. WHIRR-379.patch
        17 kB
        David Alves
      2. WHIRR-379.patch
        30 kB
        David Alves
      3. WHIRR-379.patch
        30 kB
        David Alves
      4. WHIRR-379.patch
        185 kB
        David Alves
      5. WHIRR-379.patch
        9 kB
        David Alves
      6. WHIRR-379
        5 kB
        David Alves
      7. whirr.log
        397 kB
        David Alves
      8. solo.txt
        151 kB
        Adrian Cole
      9. jclouds-wire.log
        362 kB
        David Alves
      10. jclouds-compute.log
        14 kB
        David Alves
      11. jclouds.log
        119 kB
        David Alves

        Activity

        Hide
        Frank Scholten added a comment -

        I cannot get the latest patches from march applied correctly. I checked out commit e002ebfd9f736ff5bb found via

        git rev-list --all -n 1 --first-parent --before='2012-05-23'
        

        but I end of with a lot of rejects.

        Does anyone have a recent patch for this? Or even better a Github branch?

        Show
        Frank Scholten added a comment - I cannot get the latest patches from march applied correctly. I checked out commit e002ebfd9f736ff5bb found via git rev-list --all -n 1 --first-parent --before='2012-05-23' but I end of with a lot of rejects. Does anyone have a recent patch for this? Or even better a Github branch?
        Hide
        Frank Scholten added a comment -

        @David - Any news on jclouds-virtualbox?

        Show
        Frank Scholten added a comment - @David - Any news on jclouds-virtualbox?
        Hide
        Frank Scholten added a comment -

        Same error with 1.5.0-alpha.5 and 1.5.0-alpha.4

        Show
        Frank Scholten added a comment - Same error with 1.5.0-alpha.5 and 1.5.0-alpha.4
        Hide
        Frank Scholten added a comment -

        When I use jclouds-1.5.0-alpha.6 I get the following exception immediately on startup:

        2012-05-07 13:02:09,353 ERROR [org.apache.whirr.ClusterController] (main) Unable to start the cluster. Terminating all nodes.
        com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException
        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2234)
        at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969)
        at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829)
        at com.google.common.cache.LocalCache$LocalManualCache.getUnchecked(LocalCache.java:4834)
        at org.apache.whirr.service.ComputeCache.apply(ComputeCache.java:75)
        at org.apache.whirr.service.ComputeCache.apply(ComputeCache.java:67)
        at org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:107)
        at org.apache.whirr.ClusterController.bootstrapCluster(ClusterController.java:138)
        at org.apache.whirr.ClusterController.launchCluster(ClusterController.java:114)
        at org.apache.whirr.service.zookeeper.integration.ZooKeeperServiceTest.setUp(ZooKeeperServiceTest.java:61)

        Show
        Frank Scholten added a comment - When I use jclouds-1.5.0-alpha.6 I get the following exception immediately on startup: 2012-05-07 13:02:09,353 ERROR [org.apache.whirr.ClusterController] (main) Unable to start the cluster. Terminating all nodes. com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2234) at com.google.common.cache.LocalCache.get(LocalCache.java:3965) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) at com.google.common.cache.LocalCache$LocalManualCache.getUnchecked(LocalCache.java:4834) at org.apache.whirr.service.ComputeCache.apply(ComputeCache.java:75) at org.apache.whirr.service.ComputeCache.apply(ComputeCache.java:67) at org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:107) at org.apache.whirr.ClusterController.bootstrapCluster(ClusterController.java:138) at org.apache.whirr.ClusterController.launchCluster(ClusterController.java:114) at org.apache.whirr.service.zookeeper.integration.ZooKeeperServiceTest.setUp(ZooKeeperServiceTest.java:61)
        Hide
        Frank Scholten added a comment -

        I upgraded to Virtualbox 1.4.14 and Ubuntu 12.04 and now I get a similar error:

        2012-05-05 00:33:11,627 ERROR [jclouds.compute] (user thread 1) createNodesInGroup(zookeeper-itest-virtualbox-frank), completed: 0/2, errors: 1, rate: 282406ms/op
        java.util.concurrent.ExecutionException: org.virtualbox_4_1.VBoxException: VirtualBox error: Invalid managed object reference "00da0534f69734b8-0000000000000003"
        at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:289)
        at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:276)
        at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:111)

        Show
        Frank Scholten added a comment - I upgraded to Virtualbox 1.4.14 and Ubuntu 12.04 and now I get a similar error: 2012-05-05 00:33:11,627 ERROR [jclouds.compute] (user thread 1) createNodesInGroup(zookeeper-itest-virtualbox-frank), completed: 0/2, errors: 1, rate: 282406ms/op java.util.concurrent.ExecutionException: org.virtualbox_4_1.VBoxException: VirtualBox error: Invalid managed object reference "00da0534f69734b8-0000000000000003" at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:289) at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:276) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:111)
        Hide
        David Alves added a comment -

        Hi frank

        Sorry for the late reply, linux support is still mostly untested, although I see no major reason for it not to work. I'll try to replicate the issue on my linux box and I'll get back to you. Also things have improved a lot in the latest jclouds alphas so upgrading might help. I'll try that too.

        Thanks for testing and for the useful feedback

        Show
        David Alves added a comment - Hi frank Sorry for the late reply, linux support is still mostly untested, although I see no major reason for it not to work. I'll try to replicate the issue on my linux box and I'll get back to you. Also things have improved a lot in the latest jclouds alphas so upgrading might help. I'll try that too. Thanks for testing and for the useful feedback
        Hide
        Frank Scholten added a comment -

        I started vboxwebsrv and ran the Zookeeper test. From the virtualbox GUI I could see it would install a machine, in the unit test I got SSH timeouts but it still kept on going. Then when the VM was finally booted I got the error:

        2012-05-04 23:05:19,613 ERROR [jclouds.compute] (user thread 1) createNodesInGroup(zookeeper-itest-virtualbox-frank), completed: 0/2, errors: 2, rate: 298253ms/op
        java.util.concurrent.ExecutionException: org.virtualbox_4_1.VBoxException: VirtualBox error: Invalid managed object reference "3b28b87831d58dd8-0000000000000002"

        I am running Virtualbox 1.4.8 64-bit on Ubuntu 11.10

        Show
        Frank Scholten added a comment - I started vboxwebsrv and ran the Zookeeper test. From the virtualbox GUI I could see it would install a machine, in the unit test I got SSH timeouts but it still kept on going. Then when the VM was finally booted I got the error: 2012-05-04 23:05:19,613 ERROR [jclouds.compute] (user thread 1) createNodesInGroup(zookeeper-itest-virtualbox-frank), completed: 0/2, errors: 2, rate: 298253ms/op java.util.concurrent.ExecutionException: org.virtualbox_4_1.VBoxException: VirtualBox error: Invalid managed object reference "3b28b87831d58dd8-0000000000000002" I am running Virtualbox 1.4.8 64-bit on Ubuntu 11.10
        Hide
        Frank Scholten added a comment -

        What are the steps to getting the Zookeeper VBox test passing locally on Linux? I already applied the latest patch on trunk locally.

        Show
        Frank Scholten added a comment - What are the steps to getting the Zookeeper VBox test passing locally on Linux? I already applied the latest patch on trunk locally.
        Hide
        Andrei Savu added a comment -

        vbox needs passwordless ssh access to the host (localhost)

        Works as expected. I've been able to run vbox live tests some time ago.

        starting the vbox processes is mac osx only for now, so if not on a mac could you try and start them manually

        I'm using a mac.

        Can you give me some details on the automatic configuration procedure? (I will start to dig through the code)

        Show
        Andrei Savu added a comment - vbox needs passwordless ssh access to the host (localhost) Works as expected. I've been able to run vbox live tests some time ago. starting the vbox processes is mac osx only for now, so if not on a mac could you try and start them manually I'm using a mac. Can you give me some details on the automatic configuration procedure? (I will start to dig through the code)
        Hide
        Frank Scholten added a comment - - edited

        Love to use this for testing out Whirr services like WHIRR-465.

        David: What has to be done so vbox processes also work under Linux?

        Show
        Frank Scholten added a comment - - edited Love to use this for testing out Whirr services like WHIRR-465 . David: What has to be done so vbox processes also work under Linux?
        Hide
        David Alves added a comment -

        Andrei: first of all thank you for testing we desperately need ppl to try things out

        two important things (that should be better documented so this is progress already):

        • vbox needs passwordless ssh access to the host (localhost)
        • starting the vbox processes is mac osx only for now, so if not on a mac could you try and start them manually
        Show
        David Alves added a comment - Andrei: first of all thank you for testing we desperately need ppl to try things out two important things (that should be better documented so this is progress already): vbox needs passwordless ssh access to the host (localhost) starting the vbox processes is mac osx only for now, so if not on a mac could you try and start them manually
        Hide
        Andrei Savu added a comment -

        I get no error - the launch-cluster command just blocks. The last message I see in the log file is the command starting vboxwebsrv.

        Show
        Andrei Savu added a comment - I get no error - the launch-cluster command just blocks. The last message I see in the log file is the command starting vboxwebsrv.
        Hide
        David Alves added a comment -

        Andrei are you on a mac? Could you post the error plz?

        thanks

        Show
        David Alves added a comment - Andrei are you on a mac? Could you post the error plz? thanks
        Hide
        Andrei Savu added a comment -

        I was unable to make it work just by applying the patch and running:

        ./bin/whirr launch-cluster --config recipes/zookeeper-vbox.properties
        

        It seems like vboxwebsrv starts as expected but I guess I'm missing the .iso.

        Any ideas? I will look through the virtualbox code later today.

        Show
        Andrei Savu added a comment - I was unable to make it work just by applying the patch and running: ./bin/whirr launch-cluster --config recipes/zookeeper-vbox.properties It seems like vboxwebsrv starts as expected but I guess I'm missing the .iso. Any ideas? I will look through the virtualbox code later today.
        Hide
        David Alves added a comment -

        turns out that we don't need to upgrade jclouds to 1.5.0 to make vbox work just the published virtualbox dep (1.5.0-alpha.1) is on top of WHIRR-504 is enough.

        This patch applies on top of the latest version of WHIRR-504 and is much much simpler than the previous one. Includes just a zk itest that runs against vbox and a zk vbox recipe.

        Show
        David Alves added a comment - turns out that we don't need to upgrade jclouds to 1.5.0 to make vbox work just the published virtualbox dep (1.5.0-alpha.1) is on top of WHIRR-504 is enough. This patch applies on top of the latest version of WHIRR-504 and is much much simpler than the previous one. Includes just a zk itest that runs against vbox and a zk vbox recipe.
        Hide
        Andrei Savu added a comment -

        Both jclouds and jclouds-karaf 1.5.0-alpha.1 are now available from maven central.

        Show
        Andrei Savu added a comment - Both jclouds and jclouds-karaf 1.5.0-alpha.1 are now available from maven central.
        Hide
        David Alves added a comment -

        patch includes upgrade to 1.5.0-SNAPSHOT as 1.5.0-alpha1 is not completely synced.

        Show
        David Alves added a comment - patch includes upgrade to 1.5.0-SNAPSHOT as 1.5.0-alpha1 is not completely synced.
        Hide
        David Alves added a comment -

        First patch with a vbox recipe and a zookeeper local itest. Not sure it's working (although it should) because fastdnsrsolver is failing.

        Show
        David Alves added a comment - First patch with a vbox recipe and a zookeeper local itest. Not sure it's working (although it should) because fastdnsrsolver is failing.
        Hide
        David Alves added a comment -

        I have a working implementation, runs zookeeper itest, just missing destroy. will post soon.

        Show
        David Alves added a comment - I have a working implementation, runs zookeeper itest, just missing destroy. will post soon.
        Hide
        David Alves added a comment -

        Sure Andrei (quick and dirty though)

        Show
        David Alves added a comment - Sure Andrei (quick and dirty though)
        Hide
        Andrei Savu added a comment -

        Can you collect the jclouds logs? Maybe Adrian can help.

        Show
        Andrei Savu added a comment - Can you collect the jclouds logs? Maybe Adrian can help.
        Hide
        David Alves added a comment -

        Most recent patch. Still can't log into whirr booted instances.

        launchCluster() lauches the instances (which are marked as running in openstack) but never returns.

        Show
        David Alves added a comment - Most recent patch. Still can't log into whirr booted instances. launchCluster() lauches the instances (which are marked as running in openstack) but never returns.
        Hide
        David Alves added a comment - - edited

        I'm having problems logging into instances created by whirr.

        doing an euca-describe-instances on the local cloud gives:

        RESERVATION	r-rhjoa2jk	admin	default
        INSTANCE	i-00000002	ami-59aa5f14	10.0.86.4	10.0.86.4	running	test (admin, nova)	0		m1.tiny	2011-09-04T01:55:51Z	nova		
        RESERVATION	r-say9iqm0	admin	default
        INSTANCE	i-00000001	ami-59aa5f14	10.0.86.3	10.0.86.3	running	None (admin, nova)	0		m1.tiny	2011-09-04T01:19:21Z	nova	
        

        where the *.86.3 instance was created by whirr and the *.86.4 was created manually (euca-run-instances) notice the lack of keypair in the instance launched by whirr.

        Is this supposed to happen?

        I had provided a keypair and pointed ClusterSpec to it (in order not to use temporary keys and be able to log into instances).

        Am I doing something wrong, I mean if I manually do an euca-add-keypair and then launch an instance based on that keypair I'm able to get a workable instance, which seem to point to the local-cloud thing being feasible (although still not sure useful)?

        Show
        David Alves added a comment - - edited I'm having problems logging into instances created by whirr. doing an euca-describe-instances on the local cloud gives: RESERVATION r-rhjoa2jk admin default INSTANCE i-00000002 ami-59aa5f14 10.0.86.4 10.0.86.4 running test (admin, nova) 0 m1.tiny 2011-09-04T01:55:51Z nova RESERVATION r-say9iqm0 admin default INSTANCE i-00000001 ami-59aa5f14 10.0.86.3 10.0.86.3 running None (admin, nova) 0 m1.tiny 2011-09-04T01:19:21Z nova where the *.86.3 instance was created by whirr and the *.86.4 was created manually (euca-run-instances) notice the lack of keypair in the instance launched by whirr. Is this supposed to happen? I had provided a keypair and pointed ClusterSpec to it (in order not to use temporary keys and be able to log into instances). Am I doing something wrong, I mean if I manually do an euca-add-keypair and then launch an instance based on that keypair I'm able to get a workable instance, which seem to point to the local-cloud thing being feasible (although still not sure useful)?
        Hide
        David Alves added a comment - - edited

        Some interesting steps in the right direction.

        • Made things work with whirr directly (which is now able to launch a cluster).
        • Added an url for a maverick image, i.e., a usable image can now be booted.
        • Made the Vagrantfile a template so that more than one may be used.

        Still todo:

        • start-vagrant.sh script sometimes fails to build the image cache apropriately. Workaround is to download the image file and place it at .cache/images/ url can be found on "images" file
        • still haven't tried logging into whirr launched instances (outside of whirr, e.g. by "manual" ssh).
        Show
        David Alves added a comment - - edited Some interesting steps in the right direction. Made things work with whirr directly (which is now able to launch a cluster). Added an url for a maverick image, i.e., a usable image can now be booted. Made the Vagrantfile a template so that more than one may be used. Still todo: start-vagrant.sh script sometimes fails to build the image cache apropriately. Workaround is to download the image file and place it at .cache/images/ url can be found on "images" file still haven't tried logging into whirr launched instances (outside of whirr, e.g. by "manual" ssh).
        Hide
        David Alves added a comment - - edited

        Ok, so apparently the download/registration of the test image used to launch the initial test instances in openstack failed. Will look into it and post some feedback later.

        In the mean time some times problems are solved be clearing the cache (rm -rf .cache), but I'm not sure it will work in this case.

        Show
        David Alves added a comment - - edited Ok, so apparently the download/registration of the test image used to launch the initial test instances in openstack failed. Will look into it and post some feedback later. In the mean time some times problems are solved be clearing the cache (rm -rf .cache), but I'm not sure it will work in this case.
        Hide
        Adrian Cole added a comment -

        test results

        Show
        Adrian Cole added a comment - test results
        Hide
        Adrian Cole added a comment -

        I setup my osx lion env like the following and got an error based on the attachment:

        1. setup ruby
          mkdir -p ~/.rvm/src/ && cd ~/.rvm/src && rm -rf ./rvm/ && git clone --depth 1 git://github.com/wayneeseguin/rvm.git && cd rvm && ./install
          source "/Users/adriancole/.rvm/scripts/rvm"
          echo ' [[ -s "/Users/adriancole/.rvm/scripts/rvm" ]] && source "/Users/adriancole/.rvm/scripts/rvm"' >> ~/.bashrc
          rvm pkg install openssl
          rvm pkg install readline
          rvm pkg install iconv
          rvm install 1.9.2 C "-with-openssl-dir=$HOME/.rvm/usr --with-readline-dir=$HOME/.rvm/usr --with-iconv-dir=$HOME/.rvm/usr"
          rvm --default 1.9.2
        1. install vagrant
          gem update --system
          gem install vagrant
        1. test run vagrant
          vagrant box add lucid32 http://files.vagrantup.com/lucid32.box
          vagrant init lucid32
          vagrant up
          vagrant halt
          vagrant destroy
        1. test patch
          cd /Users/adriancole/workspace/whirr
          patch -p0 --input ~/Downloads/WHIRR-379.txt
          cd contrib/local-cloud/
          chmod 755 bin/start-vagrant.sh
          bin/start-vagrant.sh
            1. failure occured

        bin/stop-vagrant.sh

        Show
        Adrian Cole added a comment - I setup my osx lion env like the following and got an error based on the attachment: setup ruby mkdir -p ~/.rvm/src/ && cd ~/.rvm/src && rm -rf ./rvm/ && git clone --depth 1 git://github.com/wayneeseguin/rvm.git && cd rvm && ./install source "/Users/adriancole/.rvm/scripts/rvm" echo ' [[ -s "/Users/adriancole/.rvm/scripts/rvm" ]] && source "/Users/adriancole/.rvm/scripts/rvm"' >> ~/.bashrc rvm pkg install openssl rvm pkg install readline rvm pkg install iconv rvm install 1.9.2 C " -with-openssl-dir=$HOME/.rvm/usr --with-readline-dir=$HOME/.rvm/usr --with-iconv-dir=$HOME/.rvm/usr" rvm --default 1.9.2 install vagrant gem update --system gem install vagrant test run vagrant vagrant box add lucid32 http://files.vagrantup.com/lucid32.box vagrant init lucid32 vagrant up vagrant halt vagrant destroy test patch cd /Users/adriancole/workspace/whirr patch -p0 --input ~/Downloads/ WHIRR-379 .txt cd contrib/local-cloud/ chmod 755 bin/start-vagrant.sh bin/start-vagrant.sh failure occured bin/stop-vagrant.sh
        Hide
        David Alves added a comment -

        After I corrected the bug in jclouds (in my gh fork) I got jclouds to start instances in the local cloud. Still not able to make it work from whirr...

        Show
        David Alves added a comment - After I corrected the bug in jclouds (in my gh fork) I got jclouds to start instances in the local cloud. Still not able to make it work from whirr...
        Hide
        David Alves added a comment -

        Adrian

        I debugged the direct jclouds test case and it turns out that the nova.domain.Image object being transformed to a compute.domain.Image does not have an updated date which is why this fails.

        Particularly...

        class NovaImageToImage
        ...
        public Image apply(org.jclouds.openstack.nova.domain.Image from) {
              ImageBuilder builder = new ImageBuilder();
              builder.ids(from.getId() + "");
              builder.name(from.getName() != null ? from.getName() : "unspecified");
              builder.description(from.getName() != null ? from.getName() : "unspecified");
         >>>     builder.version(from.getUpdated().getTime() + "");
              builder.operatingSystem(imageToOs.apply(from)); //image name may not represent the OS type
              builder.defaultCredentials(new Credentials("root", null));
              builder.uri(from.getURI());
              Image image = builder.build();
              return image;
           }
        
        Show
        David Alves added a comment - Adrian I debugged the direct jclouds test case and it turns out that the nova.domain.Image object being transformed to a compute.domain.Image does not have an updated date which is why this fails. Particularly... class NovaImageToImage ... public Image apply(org.jclouds.openstack.nova.domain.Image from) { ImageBuilder builder = new ImageBuilder(); builder.ids(from.getId() + ""); builder.name(from.getName() != null ? from.getName() : "unspecified" ); builder.description(from.getName() != null ? from.getName() : "unspecified" ); >>> builder.version(from.getUpdated().getTime() + ""); builder.operatingSystem(imageToOs.apply(from)); //image name may not represent the OS type builder.defaultCredentials( new Credentials( "root" , null )); builder.uri(from.getURI()); Image image = builder.build(); return image; }
        Hide
        David Alves added a comment -

        Any ideas?

        Show
        David Alves added a comment - Any ideas?
        Hide
        David Alves added a comment -

        Testing from whirr with:

        @Test
          public void testInteractWithLocalCloud() throws Exception {
            ClusterSpec spec = ClusterSpec.withTemporaryKeys(config);
            ClusterController controller = new ClusterController();
            Cluster cluster = controller.launchCluster(spec);
          }
        

        and with config:
        Key: whirr.cluster-name Value: chefclustertest
        Key: whirr.instance-templates Value: 1 chef
        Key: whirr.provider Value: nova
        Key: whirr.hardware-min-ram Value: 512
        Key: whirr.credential Value: mM0VdtAkFLmNYQyDCp77
        Key: whirr.identity Value: admin
        Key: jclouds.endpoint Value: http://192.168.86.100:8774/v1.0/
        Key: whirr.public.key.file Value: pk.pem

        I get:

        java.lang.NullPointerException
        at org.jclouds.openstack.nova.compute.functions.NovaImageToImage.apply(NovaImageToImage.java:49)
        at org.jclouds.openstack.nova.compute.functions.NovaImageToImage.apply(NovaImageToImage.java:35)
        at com.google.common.collect.Iterators$8.next(Iterators.java:765)
        at com.google.common.collect.Sets.newLinkedHashSet(Sets.java:267)
        at org.jclouds.openstack.nova.compute.suppliers.NovaImageSupplier.get(NovaImageSupplier.java:65)
        at org.jclouds.openstack.nova.compute.suppliers.NovaImageSupplier.get(NovaImageSupplier.java:44)
        at org.jclouds.compute.config.BaseComputeServiceContextModule$8.get(BaseComputeServiceContextModule.java:217)
        at org.jclouds.compute.config.BaseComputeServiceContextModule$8.get(BaseComputeServiceContextModule.java:214)
        at org.jclouds.rest.suppliers.SetAndThrowAuthorizationExceptionSupplier.get(SetAndThrowAuthorizationExceptionSupplier.java:49)
        at org.jclouds.concurrent.RetryOnTimeOutExceptionSupplier.get(RetryOnTimeOutExceptionSupplier.java:47)
        at com.google.common.base.Suppliers$ExpiringMemoizingSupplier.get(Suppliers.java:173)
        at org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.get(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:59)
        at org.jclouds.compute.domain.internal.TemplateBuilderImpl.getImages(TemplateBuilderImpl.java:669)
        at org.jclouds.compute.domain.internal.TemplateBuilderImpl.build(TemplateBuilderImpl.java:562)
        at org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:245)
        at org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:250)
        at org.apache.whirr.local.LocalCloudBootstrapTest.testJClouds(LocalCloudBootstrapTest.java:69)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
        at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
        at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)

        Show
        David Alves added a comment - Testing from whirr with: @Test public void testInteractWithLocalCloud() throws Exception { ClusterSpec spec = ClusterSpec.withTemporaryKeys(config); ClusterController controller = new ClusterController(); Cluster cluster = controller.launchCluster(spec); } and with config: Key: whirr.cluster-name Value: chefclustertest Key: whirr.instance-templates Value: 1 chef Key: whirr.provider Value: nova Key: whirr.hardware-min-ram Value: 512 Key: whirr.credential Value: mM0VdtAkFLmNYQyDCp77 Key: whirr.identity Value: admin Key: jclouds.endpoint Value: http://192.168.86.100:8774/v1.0/ Key: whirr.public.key.file Value: pk.pem I get: java.lang.NullPointerException at org.jclouds.openstack.nova.compute.functions.NovaImageToImage.apply(NovaImageToImage.java:49) at org.jclouds.openstack.nova.compute.functions.NovaImageToImage.apply(NovaImageToImage.java:35) at com.google.common.collect.Iterators$8.next(Iterators.java:765) at com.google.common.collect.Sets.newLinkedHashSet(Sets.java:267) at org.jclouds.openstack.nova.compute.suppliers.NovaImageSupplier.get(NovaImageSupplier.java:65) at org.jclouds.openstack.nova.compute.suppliers.NovaImageSupplier.get(NovaImageSupplier.java:44) at org.jclouds.compute.config.BaseComputeServiceContextModule$8.get(BaseComputeServiceContextModule.java:217) at org.jclouds.compute.config.BaseComputeServiceContextModule$8.get(BaseComputeServiceContextModule.java:214) at org.jclouds.rest.suppliers.SetAndThrowAuthorizationExceptionSupplier.get(SetAndThrowAuthorizationExceptionSupplier.java:49) at org.jclouds.concurrent.RetryOnTimeOutExceptionSupplier.get(RetryOnTimeOutExceptionSupplier.java:47) at com.google.common.base.Suppliers$ExpiringMemoizingSupplier.get(Suppliers.java:173) at org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.get(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:59) at org.jclouds.compute.domain.internal.TemplateBuilderImpl.getImages(TemplateBuilderImpl.java:669) at org.jclouds.compute.domain.internal.TemplateBuilderImpl.build(TemplateBuilderImpl.java:562) at org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:245) at org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:250) at org.apache.whirr.local.LocalCloudBootstrapTest.testJClouds(LocalCloudBootstrapTest.java:69) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
        Hide
        David Alves added a comment -

        Testing solely with jclouds with:

        Bar.java
        @Test
          public void testJClouds() throws Exception {
            Properties overrides = new Properties();
            // point the JClouds to the Openstack v 1.1 API endpoint URL
            overrides.setProperty(Constants.PROPERTY_ENDPOINT, "http://192.168.86.100:8774");
        
            // get a context with nova that offers the portable ComputeService api
            ComputeServiceContext context = new ComputeServiceContextFactory().createContext("nova", "admin",
                "mM0VdtAkFLmNYQyDCp77", ImmutableSet.<Module> of(new SshjSshClientModule()), overrides);
        
            // run some nodes
            Set<? extends NodeMetadata> nodes = context.getComputeService().createNodesInGroup("webserver", 2);
          }
        

        I get the error:

        java.lang.NullPointerException
        at org.jclouds.openstack.nova.compute.functions.NovaImageToImage.apply(NovaImageToImage.java:49)
        at org.jclouds.openstack.nova.compute.functions.NovaImageToImage.apply(NovaImageToImage.java:35)
        at com.google.common.collect.Iterators$8.next(Iterators.java:765)
        at com.google.common.collect.Sets.newLinkedHashSet(Sets.java:267)
        at org.jclouds.openstack.nova.compute.suppliers.NovaImageSupplier.get(NovaImageSupplier.java:65)
        at org.jclouds.openstack.nova.compute.suppliers.NovaImageSupplier.get(NovaImageSupplier.java:44)
        at org.jclouds.compute.config.BaseComputeServiceContextModule$8.get(BaseComputeServiceContextModule.java:217)
        at org.jclouds.compute.config.BaseComputeServiceContextModule$8.get(BaseComputeServiceContextModule.java:214)
        at org.jclouds.rest.suppliers.SetAndThrowAuthorizationExceptionSupplier.get(SetAndThrowAuthorizationExceptionSupplier.java:49)
        at org.jclouds.concurrent.RetryOnTimeOutExceptionSupplier.get(RetryOnTimeOutExceptionSupplier.java:47)
        at com.google.common.base.Suppliers$ExpiringMemoizingSupplier.get(Suppliers.java:173)
        at org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.get(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:59)
        at org.jclouds.compute.domain.internal.TemplateBuilderImpl.getImages(TemplateBuilderImpl.java:669)
        at org.jclouds.compute.domain.internal.TemplateBuilderImpl.build(TemplateBuilderImpl.java:562)
        at org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:245)
        at org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:250)
        at org.apache.whirr.local.LocalCloudBootstrapTest.testJClouds(LocalCloudBootstrapTest.java:69)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
        at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
        at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)

        Show
        David Alves added a comment - Testing solely with jclouds with: Bar.java @Test public void testJClouds() throws Exception { Properties overrides = new Properties(); // point the JClouds to the Openstack v 1.1 API endpoint URL overrides.setProperty(Constants.PROPERTY_ENDPOINT, "http: //192.168.86.100:8774" ); // get a context with nova that offers the portable ComputeService api ComputeServiceContext context = new ComputeServiceContextFactory().createContext( "nova" , "admin" , "mM0VdtAkFLmNYQyDCp77" , ImmutableSet.<Module> of( new SshjSshClientModule()), overrides); // run some nodes Set<? extends NodeMetadata> nodes = context.getComputeService().createNodesInGroup( "webserver" , 2); } I get the error: java.lang.NullPointerException at org.jclouds.openstack.nova.compute.functions.NovaImageToImage.apply(NovaImageToImage.java:49) at org.jclouds.openstack.nova.compute.functions.NovaImageToImage.apply(NovaImageToImage.java:35) at com.google.common.collect.Iterators$8.next(Iterators.java:765) at com.google.common.collect.Sets.newLinkedHashSet(Sets.java:267) at org.jclouds.openstack.nova.compute.suppliers.NovaImageSupplier.get(NovaImageSupplier.java:65) at org.jclouds.openstack.nova.compute.suppliers.NovaImageSupplier.get(NovaImageSupplier.java:44) at org.jclouds.compute.config.BaseComputeServiceContextModule$8.get(BaseComputeServiceContextModule.java:217) at org.jclouds.compute.config.BaseComputeServiceContextModule$8.get(BaseComputeServiceContextModule.java:214) at org.jclouds.rest.suppliers.SetAndThrowAuthorizationExceptionSupplier.get(SetAndThrowAuthorizationExceptionSupplier.java:49) at org.jclouds.concurrent.RetryOnTimeOutExceptionSupplier.get(RetryOnTimeOutExceptionSupplier.java:47) at com.google.common.base.Suppliers$ExpiringMemoizingSupplier.get(Suppliers.java:173) at org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.get(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:59) at org.jclouds.compute.domain.internal.TemplateBuilderImpl.getImages(TemplateBuilderImpl.java:669) at org.jclouds.compute.domain.internal.TemplateBuilderImpl.build(TemplateBuilderImpl.java:562) at org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:245) at org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:250) at org.apache.whirr.local.LocalCloudBootstrapTest.testJClouds(LocalCloudBootstrapTest.java:69) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
        Hide
        David Alves added a comment -

        Made some progress.

        Cloud bootup and image registry is almost done.

        Created a cache for images (otherwise would have to download everytime)

        Started tests, this is where I hit a roadblock.

        Show
        David Alves added a comment - Made some progress. Cloud bootup and image registry is almost done. Created a cache for images (otherwise would have to download everytime) Started tests, this is where I hit a roadblock.
        Hide
        David Alves added a comment -

        Next steps will be to try and interact with it through whirr, and making common images available, such as CentOS and Ubuntu (only tty for now)

        Show
        David Alves added a comment - Next steps will be to try and interact with it through whirr, and making common images available, such as CentOS and Ubuntu (only tty for now)
        Hide
        David Alves added a comment -

        Ok, here we go.

        This patch adds a vagrant config file and some simple bash scripts to start things up.

        I had already installed virtualbox and vagrant and chef so I would kindly ask someone else to document this.

        After the pre-requisites are met testing your own openstack cloud should be as simple as:

        bin/start-vagrant.sh

        vagrant ssh

        . /vagrant/novarc
        euca-add-keypair test > test.pem
        chmod 600 test.pem
        euca-run-instances -t m1.tiny -k test ami-tty

        1. wait for boot (euca-describe-instances should report running)
          ssh -i test.pem root@10.0.0.3
        Show
        David Alves added a comment - Ok, here we go. This patch adds a vagrant config file and some simple bash scripts to start things up. I had already installed virtualbox and vagrant and chef so I would kindly ask someone else to document this. After the pre-requisites are met testing your own openstack cloud should be as simple as: bin/start-vagrant.sh vagrant ssh . /vagrant/novarc euca-add-keypair test > test.pem chmod 600 test.pem euca-run-instances -t m1.tiny -k test ami-tty wait for boot (euca-describe-instances should report running) ssh -i test.pem root@10.0.0.3

          People

          • Assignee:
            David Alves
            Reporter:
            David Alves
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development