Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: core
    • Labels:
      None

      Description

      We should upgrade to the latest jclouds release when it comes out. This includes support for Eucalyptus as well a number of fixes to EC2.

      1. WHIRR-124-warning.patch
        2 kB
        Adrian Cole
      2. WHIRR-124.patch
        5 kB
        Tom White
      3. WHIRR-124.patch
        6 kB
        Adrian Cole
      4. WHIRR-124.patch
        44 kB
        Adrian Cole
      5. WHIRR-124.patch
        40 kB
        Adrian Cole
      6. WHIRR-124.patch
        40 kB
        Adrian Cole

        Issue Links

          Activity

          Gavin made changes -
          Link This issue is depended upon by WHIRR-125 [ WHIRR-125 ]
          Gavin made changes -
          Link This issue blocks WHIRR-125 [ WHIRR-125 ]
          Andrei Savu made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Andrei Savu added a comment -

          +1 I've just committed the new patch. Thanks Adrian!

          Show
          Andrei Savu added a comment - +1 I've just committed the new patch. Thanks Adrian!
          Adrian Cole made changes -
          Status Reopened [ 4 ] Patch Available [ 10002 ]
          Adrian Cole made changes -
          Attachment WHIRR-124-warning.patch [ 12470530 ]
          Hide
          Adrian Cole added a comment -

          this patch warns before converting ec2->aws-ec2 and cloudservers -> clouservers-us

          tested with cassandra on both ec2 and cloudservers

          Show
          Adrian Cole added a comment - this patch warns before converting ec2->aws-ec2 and cloudservers -> clouservers-us tested with cassandra on both ec2 and cloudservers
          Hide
          Adrian Cole added a comment -

          note this is the warning message one will get when using the old provider keys:

          2011-02-07 16:27:02,426 WARN [org.apache.whirr.service.ComputeServiceContextBuilder] (main) please use provider "aws-ec2" instead of "ec2"

          Show
          Adrian Cole added a comment - note this is the warning message one will get when using the old provider keys: 2011-02-07 16:27:02,426 WARN [org.apache.whirr.service.ComputeServiceContextBuilder] (main) please use provider "aws-ec2" instead of "ec2"
          Adrian Cole made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hide
          Adrian Cole added a comment -

          opening to add another patch

          warn and convert the following providers. note that this patch should be reverted after 1 release:

          ec2 -> aws-ec2
          cloudservers -> cloudservers-us

          Show
          Adrian Cole added a comment - opening to add another patch warn and convert the following providers. note that this patch should be reverted after 1 release: ec2 -> aws-ec2 cloudservers -> cloudservers-us
          Andrei Savu made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Andrei Savu added a comment -

          I've just re-committed this after re-running all the tests. I'm seeing only one failure in the CDH integration tests but I believe it's not related to this patch and we should fix it in WHIRR-227. Thanks Adrian and Lars and sorry for all the trouble around this.

          Show
          Andrei Savu added a comment - I've just re-committed this after re-running all the tests. I'm seeing only one failure in the CDH integration tests but I believe it's not related to this patch and we should fix it in WHIRR-227 . Thanks Adrian and Lars and sorry for all the trouble around this.
          Adrian Cole made changes -
          Attachment WHIRR-124.patch [ 12470506 ]
          Hide
          Adrian Cole added a comment -

          reworked to sync up with trunk's latest line numbers

          Show
          Adrian Cole added a comment - reworked to sync up with trunk's latest line numbers
          Hide
          Adrian Cole added a comment -

          -1

          your environment needs to use the aws-ec2 provider key. Please don't roll back changes because a private environment fails. If you plan to, at least note in the case the issue you encountered.

          Show
          Adrian Cole added a comment - -1 your environment needs to use the aws-ec2 provider key. Please don't roll back changes because a private environment fails. If you plan to, at least note in the case the issue you encountered.
          Andrei Savu made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hide
          Andrei Savu added a comment -

          I've just reverted the change.

          Show
          Andrei Savu added a comment - I've just reverted the change.
          Hide
          Tom White added a comment -

          Per comments in WHIRR-227, integration tests are now failing because of this change. Let's revert this change until we can figure out what's broken.

          Show
          Tom White added a comment - Per comments in WHIRR-227 , integration tests are now failing because of this change. Let's revert this change until we can figure out what's broken.
          Hide
          Patrick Hunt added a comment -

          My bad, for some reason I was under the impression 0.4.0 was already out there.

          Show
          Patrick Hunt added a comment - My bad, for some reason I was under the impression 0.4.0 was already out there.
          Hide
          Patrick Hunt added a comment -

          this is marked fixed in 0.4.0, but isn't it affecting 0.4.0 and fixed in upcoming 0.5.0?

          Show
          Patrick Hunt added a comment - this is marked fixed in 0.4.0, but isn't it affecting 0.4.0 and fixed in upcoming 0.5.0?
          Hide
          Tom White added a comment -

          I uploaded the scripts to S3.

          Show
          Tom White added a comment - I uploaded the scripts to S3.
          Lars George made changes -
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.4.0 [ 12316065 ]
          Hide
          Lars George added a comment -

          +1

          Committed to trunk. Thanks Adrian for the awesome work!

          Show
          Lars George added a comment - +1 Committed to trunk. Thanks Adrian for the awesome work!
          Adrian Cole made changes -
          Attachment WHIRR-124.patch [ 12470422 ]
          Hide
          Adrian Cole added a comment -

          note you have to update the whirr scripts with what's in this patch!

          tested with cassandra, hadoop, hbase, zookeeper

          Show
          Adrian Cole added a comment - note you have to update the whirr scripts with what's in this patch! tested with cassandra, hadoop, hbase, zookeeper
          Hide
          Tom White added a comment -

          The post-configure scripts should be OK now. See WHIRR-227.

          Show
          Tom White added a comment - The post-configure scripts should be OK now. See WHIRR-227 .
          Hide
          Adrian Cole added a comment -

          I've confirmed that a clean checkout of whirr fails on hadoop tests, likely due to the shortnames in the post-configure script.

          Hopefully, once the post-configure scripts are sorted, the patch already attached to this case will pass cleanly.

          Show
          Adrian Cole added a comment - I've confirmed that a clean checkout of whirr fails on hadoop tests, likely due to the shortnames in the post-configure script. Hopefully, once the post-configure scripts are sorted, the patch already attached to this case will pass cleanly.
          Adrian Cole made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Hide
          Adrian Cole added a comment -

          this includes a workaround for the runscripts that hard-code in the provider key "ec2". Note that hadoop currently fails as the roles passed in code are long names, while the runscripts use short names. ex. http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/hadoop/post-configure

          Show
          Adrian Cole added a comment - this includes a workaround for the runscripts that hard-code in the provider key "ec2". Note that hadoop currently fails as the roles passed in code are long names, while the runscripts use short names. ex. http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/hadoop/post-configure
          Adrian Cole made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Adrian Cole made changes -
          Attachment WHIRR-124.patch [ 12470409 ]
          Hide
          Adrian Cole added a comment -

          this includes a workaround for the runscripts that hard-code in the provider key "ec2". Note that hadoop currently fails as the roles passed in code are long names, while the runscripts use short names. ex. http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/hadoop/post-configure

          Show
          Adrian Cole added a comment - this includes a workaround for the runscripts that hard-code in the provider key "ec2". Note that hadoop currently fails as the roles passed in code are long names, while the runscripts use short names. ex. http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/hadoop/post-configure
          Hide
          Adrian Cole added a comment -

          another pattern:

          If you want to code based on EC2-specific features...

          instead of:
          if (clusterSpec.getProvider().equals("aws-ec2")) {

          do:
          if (computeServiceContext.getProviderSpecificContext().getApi() instanceof
          EC2Client) {

          Show
          Adrian Cole added a comment - another pattern: If you want to code based on EC2-specific features... instead of: if (clusterSpec.getProvider().equals("aws-ec2")) { do: if (computeServiceContext.getProviderSpecificContext().getApi() instanceof EC2Client) {
          Hide
          Adrian Cole added a comment -

          runscripts need to be enhanced to not depend on provider keys as much as possible. For example, amazon's ec2 is now the provider key "aws-ec2" and this makes the following script not work: http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/zookeeper/post-configure

          It would be better to make the post-configure scripts a bit smarter, as providers like e
          "eucalyptus" or "eucalyptus-partnercloud-ec2" also may have operating system access to instance metadata.

          ex, if we want to check for metadata support, we can check existence of the url path http://169.254.169.254/latest/meta-data/local-ipv4 (ex. nc -z -w 1 169.254.169.254 80)

          Show
          Adrian Cole added a comment - runscripts need to be enhanced to not depend on provider keys as much as possible. For example, amazon's ec2 is now the provider key "aws-ec2" and this makes the following script not work: http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/zookeeper/post-configure It would be better to make the post-configure scripts a bit smarter, as providers like e "eucalyptus" or "eucalyptus-partnercloud-ec2" also may have operating system access to instance metadata. ex, if we want to check for metadata support, we can check existence of the url path http://169.254.169.254/latest/meta-data/local-ipv4 (ex. nc -z -w 1 169.254.169.254 80)
          Adrian Cole made changes -
          Summary Upgrade to jclouds 1.0-beta-8 Upgrade to jclouds 1.0-beta-9
          Description We should upgrade to the latest jclouds release when it comes out. We should upgrade to the latest jclouds release when it comes out. This includes support for Eucalyptus as well a number of fixes to EC2.
          Assignee Tom White [ tomwhite ] Adrian Cole [ adrian@jclouds.org ]
          Adrian Cole made changes -
          Attachment WHIRR-124.patch [ 12460302 ]
          Adrian Cole made changes -
          Attachment WHIRR-124.patch [ 12459799 ]
          Adrian Cole made changes -
          Attachment WHIRR-124.patch [ 12459799 ]
          Hide
          Adrian Cole added a comment -

          Here's the patch with the real version. tested with hadoop, zk, and cassandra

          Show
          Adrian Cole added a comment - Here's the patch with the real version. tested with hadoop, zk, and cassandra
          Tom White made changes -
          Link This issue blocks WHIRR-125 [ WHIRR-125 ]
          Tom White made changes -
          Assignee Tom White [ tomwhite ]
          Tom White made changes -
          Field Original Value New Value
          Attachment WHIRR-124.patch [ 12458656 ]
          Hide
          Tom White added a comment -

          Here's a patch for jclouds 1.0-SNAPSHOT. Not for commit but shows the changes that will be needed for 1.0-beta-8.

          Show
          Tom White added a comment - Here's a patch for jclouds 1.0-SNAPSHOT. Not for commit but shows the changes that will be needed for 1.0-beta-8.
          Tom White created issue -

            People

            • Assignee:
              Adrian Cole
              Reporter:
              Tom White
            • Votes:
              1 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development