Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: core
    • Labels:
      None

      Description

      We should upgrade to the latest jclouds release when it comes out. This includes support for Eucalyptus as well a number of fixes to EC2.

      1. WHIRR-124.patch
        5 kB
        Tom White
      2. WHIRR-124.patch
        6 kB
        Adrian Cole
      3. WHIRR-124.patch
        44 kB
        Adrian Cole
      4. WHIRR-124.patch
        40 kB
        Adrian Cole
      5. WHIRR-124.patch
        40 kB
        Adrian Cole
      6. WHIRR-124-warning.patch
        2 kB
        Adrian Cole

        Issue Links

          Activity

          Tom White created issue -
          Hide
          Tom White added a comment -

          Here's a patch for jclouds 1.0-SNAPSHOT. Not for commit but shows the changes that will be needed for 1.0-beta-8.

          Show
          Tom White added a comment - Here's a patch for jclouds 1.0-SNAPSHOT. Not for commit but shows the changes that will be needed for 1.0-beta-8.
          Tom White made changes -
          Field Original Value New Value
          Attachment WHIRR-124.patch [ 12458656 ]
          Tom White made changes -
          Assignee Tom White [ tomwhite ]
          Tom White made changes -
          Link This issue blocks WHIRR-125 [ WHIRR-125 ]
          Hide
          Adrian Cole (Inactive) added a comment -

          Here's the patch with the real version. tested with hadoop, zk, and cassandra

          Show
          Adrian Cole (Inactive) added a comment - Here's the patch with the real version. tested with hadoop, zk, and cassandra
          Adrian Cole (Inactive) made changes -
          Attachment WHIRR-124.patch [ 12459799 ]
          Adrian Cole (Inactive) made changes -
          Attachment WHIRR-124.patch [ 12459799 ]
          Adrian Cole (Inactive) made changes -
          Attachment WHIRR-124.patch [ 12460302 ]
          Adrian Cole (Inactive) made changes -
          Summary Upgrade to jclouds 1.0-beta-8 Upgrade to jclouds 1.0-beta-9
          Description We should upgrade to the latest jclouds release when it comes out. We should upgrade to the latest jclouds release when it comes out. This includes support for Eucalyptus as well a number of fixes to EC2.
          Assignee Tom White [ tomwhite ] Adrian Cole [ adrian@jclouds.org ]
          Hide
          Adrian Cole (Inactive) added a comment -

          runscripts need to be enhanced to not depend on provider keys as much as possible. For example, amazon's ec2 is now the provider key "aws-ec2" and this makes the following script not work: http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/zookeeper/post-configure

          It would be better to make the post-configure scripts a bit smarter, as providers like e
          "eucalyptus" or "eucalyptus-partnercloud-ec2" also may have operating system access to instance metadata.

          ex, if we want to check for metadata support, we can check existence of the url path http://169.254.169.254/latest/meta-data/local-ipv4 (ex. nc -z -w 1 169.254.169.254 80)

          Show
          Adrian Cole (Inactive) added a comment - runscripts need to be enhanced to not depend on provider keys as much as possible. For example, amazon's ec2 is now the provider key "aws-ec2" and this makes the following script not work: http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/zookeeper/post-configure It would be better to make the post-configure scripts a bit smarter, as providers like e "eucalyptus" or "eucalyptus-partnercloud-ec2" also may have operating system access to instance metadata. ex, if we want to check for metadata support, we can check existence of the url path http://169.254.169.254/latest/meta-data/local-ipv4 (ex. nc -z -w 1 169.254.169.254 80)
          Hide
          Adrian Cole (Inactive) added a comment -

          another pattern:

          If you want to code based on EC2-specific features...

          instead of:
          if (clusterSpec.getProvider().equals("aws-ec2")) {

          do:
          if (computeServiceContext.getProviderSpecificContext().getApi() instanceof
          EC2Client) {

          Show
          Adrian Cole (Inactive) added a comment - another pattern: If you want to code based on EC2-specific features... instead of: if (clusterSpec.getProvider().equals("aws-ec2")) { do: if (computeServiceContext.getProviderSpecificContext().getApi() instanceof EC2Client) {
          Hide
          Adrian Cole (Inactive) added a comment -

          this includes a workaround for the runscripts that hard-code in the provider key "ec2". Note that hadoop currently fails as the roles passed in code are long names, while the runscripts use short names. ex. http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/hadoop/post-configure

          Show
          Adrian Cole (Inactive) added a comment - this includes a workaround for the runscripts that hard-code in the provider key "ec2". Note that hadoop currently fails as the roles passed in code are long names, while the runscripts use short names. ex. http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/hadoop/post-configure
          Adrian Cole (Inactive) made changes -
          Attachment WHIRR-124.patch [ 12470409 ]
          Adrian Cole (Inactive) made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Hide
          Adrian Cole (Inactive) added a comment -

          this includes a workaround for the runscripts that hard-code in the provider key "ec2". Note that hadoop currently fails as the roles passed in code are long names, while the runscripts use short names. ex. http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/hadoop/post-configure

          Show
          Adrian Cole (Inactive) added a comment - this includes a workaround for the runscripts that hard-code in the provider key "ec2". Note that hadoop currently fails as the roles passed in code are long names, while the runscripts use short names. ex. http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/hadoop/post-configure
          Adrian Cole (Inactive) made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Hide
          Adrian Cole (Inactive) added a comment -

          I've confirmed that a clean checkout of whirr fails on hadoop tests, likely due to the shortnames in the post-configure script.

          Hopefully, once the post-configure scripts are sorted, the patch already attached to this case will pass cleanly.

          Show
          Adrian Cole (Inactive) added a comment - I've confirmed that a clean checkout of whirr fails on hadoop tests, likely due to the shortnames in the post-configure script. Hopefully, once the post-configure scripts are sorted, the patch already attached to this case will pass cleanly.
          Hide
          Tom White added a comment -

          The post-configure scripts should be OK now. See WHIRR-227.

          Show
          Tom White added a comment - The post-configure scripts should be OK now. See WHIRR-227 .
          Hide
          Adrian Cole (Inactive) added a comment -

          note you have to update the whirr scripts with what's in this patch!

          tested with cassandra, hadoop, hbase, zookeeper

          Show
          Adrian Cole (Inactive) added a comment - note you have to update the whirr scripts with what's in this patch! tested with cassandra, hadoop, hbase, zookeeper
          Adrian Cole (Inactive) made changes -
          Attachment WHIRR-124.patch [ 12470422 ]
          Hide
          Lars George added a comment -

          +1

          Committed to trunk. Thanks Adrian for the awesome work!

          Show
          Lars George added a comment - +1 Committed to trunk. Thanks Adrian for the awesome work!
          Lars George made changes -
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.4.0 [ 12316065 ]
          Hide
          Tom White added a comment -

          I uploaded the scripts to S3.

          Show
          Tom White added a comment - I uploaded the scripts to S3.
          Hide
          Patrick Hunt added a comment -

          this is marked fixed in 0.4.0, but isn't it affecting 0.4.0 and fixed in upcoming 0.5.0?

          Show
          Patrick Hunt added a comment - this is marked fixed in 0.4.0, but isn't it affecting 0.4.0 and fixed in upcoming 0.5.0?
          Hide
          Patrick Hunt added a comment -

          My bad, for some reason I was under the impression 0.4.0 was already out there.

          Show
          Patrick Hunt added a comment - My bad, for some reason I was under the impression 0.4.0 was already out there.
          Hide
          Tom White added a comment -

          Per comments in WHIRR-227, integration tests are now failing because of this change. Let's revert this change until we can figure out what's broken.

          Show
          Tom White added a comment - Per comments in WHIRR-227 , integration tests are now failing because of this change. Let's revert this change until we can figure out what's broken.
          Hide
          Andrei Savu added a comment -

          I've just reverted the change.

          Show
          Andrei Savu added a comment - I've just reverted the change.
          Andrei Savu made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hide
          Adrian Cole (Inactive) added a comment -

          -1

          your environment needs to use the aws-ec2 provider key. Please don't roll back changes because a private environment fails. If you plan to, at least note in the case the issue you encountered.

          Show
          Adrian Cole (Inactive) added a comment - -1 your environment needs to use the aws-ec2 provider key. Please don't roll back changes because a private environment fails. If you plan to, at least note in the case the issue you encountered.
          Hide
          Adrian Cole (Inactive) added a comment -

          reworked to sync up with trunk's latest line numbers

          Show
          Adrian Cole (Inactive) added a comment - reworked to sync up with trunk's latest line numbers
          Adrian Cole (Inactive) made changes -
          Attachment WHIRR-124.patch [ 12470506 ]
          Hide
          Andrei Savu added a comment -

          I've just re-committed this after re-running all the tests. I'm seeing only one failure in the CDH integration tests but I believe it's not related to this patch and we should fix it in WHIRR-227. Thanks Adrian and Lars and sorry for all the trouble around this.

          Show
          Andrei Savu added a comment - I've just re-committed this after re-running all the tests. I'm seeing only one failure in the CDH integration tests but I believe it's not related to this patch and we should fix it in WHIRR-227 . Thanks Adrian and Lars and sorry for all the trouble around this.
          Andrei Savu made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Adrian Cole (Inactive) added a comment -

          opening to add another patch

          warn and convert the following providers. note that this patch should be reverted after 1 release:

          ec2 -> aws-ec2
          cloudservers -> cloudservers-us

          Show
          Adrian Cole (Inactive) added a comment - opening to add another patch warn and convert the following providers. note that this patch should be reverted after 1 release: ec2 -> aws-ec2 cloudservers -> cloudservers-us
          Adrian Cole (Inactive) made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hide
          Adrian Cole (Inactive) added a comment -

          note this is the warning message one will get when using the old provider keys:

          2011-02-07 16:27:02,426 WARN [org.apache.whirr.service.ComputeServiceContextBuilder] (main) please use provider "aws-ec2" instead of "ec2"

          Show
          Adrian Cole (Inactive) added a comment - note this is the warning message one will get when using the old provider keys: 2011-02-07 16:27:02,426 WARN [org.apache.whirr.service.ComputeServiceContextBuilder] (main) please use provider "aws-ec2" instead of "ec2"
          Hide
          Adrian Cole (Inactive) added a comment -

          this patch warns before converting ec2->aws-ec2 and cloudservers -> clouservers-us

          tested with cassandra on both ec2 and cloudservers

          Show
          Adrian Cole (Inactive) added a comment - this patch warns before converting ec2->aws-ec2 and cloudservers -> clouservers-us tested with cassandra on both ec2 and cloudservers
          Adrian Cole (Inactive) made changes -
          Attachment WHIRR-124-warning.patch [ 12470530 ]
          Adrian Cole (Inactive) made changes -
          Status Reopened [ 4 ] Patch Available [ 10002 ]
          Hide
          Andrei Savu added a comment -

          +1 I've just committed the new patch. Thanks Adrian!

          Show
          Andrei Savu added a comment - +1 I've just committed the new patch. Thanks Adrian!
          Andrei Savu made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Gavin made changes -
          Link This issue blocks WHIRR-125 [ WHIRR-125 ]
          Gavin made changes -
          Link This issue is depended upon by WHIRR-125 [ WHIRR-125 ]

            People

            • Assignee:
              Adrian Cole (Inactive)
              Reporter:
              Tom White
            • Votes:
              1 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development