Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: core
    • Labels:
      None

      Description

      We should upgrade to the latest jclouds release when it comes out. This includes support for Eucalyptus as well a number of fixes to EC2.

      1. WHIRR-124-warning.patch
        2 kB
        Adrian Cole
      2. WHIRR-124.patch
        40 kB
        Adrian Cole
      3. WHIRR-124.patch
        40 kB
        Adrian Cole
      4. WHIRR-124.patch
        44 kB
        Adrian Cole
      5. WHIRR-124.patch
        6 kB
        Adrian Cole
      6. WHIRR-124.patch
        5 kB
        Tom White

        Issue Links

          Activity

          Hide
          Tom White added a comment -

          Here's a patch for jclouds 1.0-SNAPSHOT. Not for commit but shows the changes that will be needed for 1.0-beta-8.

          Show
          Tom White added a comment - Here's a patch for jclouds 1.0-SNAPSHOT. Not for commit but shows the changes that will be needed for 1.0-beta-8.
          Hide
          Adrian Cole added a comment -

          Here's the patch with the real version. tested with hadoop, zk, and cassandra

          Show
          Adrian Cole added a comment - Here's the patch with the real version. tested with hadoop, zk, and cassandra
          Hide
          Adrian Cole added a comment -

          runscripts need to be enhanced to not depend on provider keys as much as possible. For example, amazon's ec2 is now the provider key "aws-ec2" and this makes the following script not work: http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/zookeeper/post-configure

          It would be better to make the post-configure scripts a bit smarter, as providers like e
          "eucalyptus" or "eucalyptus-partnercloud-ec2" also may have operating system access to instance metadata.

          ex, if we want to check for metadata support, we can check existence of the url path http://169.254.169.254/latest/meta-data/local-ipv4 (ex. nc -z -w 1 169.254.169.254 80)

          Show
          Adrian Cole added a comment - runscripts need to be enhanced to not depend on provider keys as much as possible. For example, amazon's ec2 is now the provider key "aws-ec2" and this makes the following script not work: http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/zookeeper/post-configure It would be better to make the post-configure scripts a bit smarter, as providers like e "eucalyptus" or "eucalyptus-partnercloud-ec2" also may have operating system access to instance metadata. ex, if we want to check for metadata support, we can check existence of the url path http://169.254.169.254/latest/meta-data/local-ipv4 (ex. nc -z -w 1 169.254.169.254 80)
          Hide
          Adrian Cole added a comment -

          another pattern:

          If you want to code based on EC2-specific features...

          instead of:
          if (clusterSpec.getProvider().equals("aws-ec2")) {

          do:
          if (computeServiceContext.getProviderSpecificContext().getApi() instanceof
          EC2Client) {

          Show
          Adrian Cole added a comment - another pattern: If you want to code based on EC2-specific features... instead of: if (clusterSpec.getProvider().equals("aws-ec2")) { do: if (computeServiceContext.getProviderSpecificContext().getApi() instanceof EC2Client) {
          Hide
          Adrian Cole added a comment -

          this includes a workaround for the runscripts that hard-code in the provider key "ec2". Note that hadoop currently fails as the roles passed in code are long names, while the runscripts use short names. ex. http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/hadoop/post-configure

          Show
          Adrian Cole added a comment - this includes a workaround for the runscripts that hard-code in the provider key "ec2". Note that hadoop currently fails as the roles passed in code are long names, while the runscripts use short names. ex. http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/hadoop/post-configure
          Hide
          Adrian Cole added a comment -

          this includes a workaround for the runscripts that hard-code in the provider key "ec2". Note that hadoop currently fails as the roles passed in code are long names, while the runscripts use short names. ex. http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/hadoop/post-configure

          Show
          Adrian Cole added a comment - this includes a workaround for the runscripts that hard-code in the provider key "ec2". Note that hadoop currently fails as the roles passed in code are long names, while the runscripts use short names. ex. http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/hadoop/post-configure
          Hide
          Adrian Cole added a comment -

          I've confirmed that a clean checkout of whirr fails on hadoop tests, likely due to the shortnames in the post-configure script.

          Hopefully, once the post-configure scripts are sorted, the patch already attached to this case will pass cleanly.

          Show
          Adrian Cole added a comment - I've confirmed that a clean checkout of whirr fails on hadoop tests, likely due to the shortnames in the post-configure script. Hopefully, once the post-configure scripts are sorted, the patch already attached to this case will pass cleanly.
          Hide
          Tom White added a comment -

          The post-configure scripts should be OK now. See WHIRR-227.

          Show
          Tom White added a comment - The post-configure scripts should be OK now. See WHIRR-227 .
          Hide
          Adrian Cole added a comment -

          note you have to update the whirr scripts with what's in this patch!

          tested with cassandra, hadoop, hbase, zookeeper

          Show
          Adrian Cole added a comment - note you have to update the whirr scripts with what's in this patch! tested with cassandra, hadoop, hbase, zookeeper
          Hide
          Lars George added a comment -

          +1

          Committed to trunk. Thanks Adrian for the awesome work!

          Show
          Lars George added a comment - +1 Committed to trunk. Thanks Adrian for the awesome work!
          Hide
          Tom White added a comment -

          I uploaded the scripts to S3.

          Show
          Tom White added a comment - I uploaded the scripts to S3.
          Hide
          Patrick Hunt added a comment -

          this is marked fixed in 0.4.0, but isn't it affecting 0.4.0 and fixed in upcoming 0.5.0?

          Show
          Patrick Hunt added a comment - this is marked fixed in 0.4.0, but isn't it affecting 0.4.0 and fixed in upcoming 0.5.0?
          Hide
          Patrick Hunt added a comment -

          My bad, for some reason I was under the impression 0.4.0 was already out there.

          Show
          Patrick Hunt added a comment - My bad, for some reason I was under the impression 0.4.0 was already out there.
          Hide
          Tom White added a comment -

          Per comments in WHIRR-227, integration tests are now failing because of this change. Let's revert this change until we can figure out what's broken.

          Show
          Tom White added a comment - Per comments in WHIRR-227 , integration tests are now failing because of this change. Let's revert this change until we can figure out what's broken.
          Hide
          Andrei Savu added a comment -

          I've just reverted the change.

          Show
          Andrei Savu added a comment - I've just reverted the change.
          Hide
          Adrian Cole added a comment -

          -1

          your environment needs to use the aws-ec2 provider key. Please don't roll back changes because a private environment fails. If you plan to, at least note in the case the issue you encountered.

          Show
          Adrian Cole added a comment - -1 your environment needs to use the aws-ec2 provider key. Please don't roll back changes because a private environment fails. If you plan to, at least note in the case the issue you encountered.
          Hide
          Adrian Cole added a comment -

          reworked to sync up with trunk's latest line numbers

          Show
          Adrian Cole added a comment - reworked to sync up with trunk's latest line numbers
          Hide
          Andrei Savu added a comment -

          I've just re-committed this after re-running all the tests. I'm seeing only one failure in the CDH integration tests but I believe it's not related to this patch and we should fix it in WHIRR-227. Thanks Adrian and Lars and sorry for all the trouble around this.

          Show
          Andrei Savu added a comment - I've just re-committed this after re-running all the tests. I'm seeing only one failure in the CDH integration tests but I believe it's not related to this patch and we should fix it in WHIRR-227 . Thanks Adrian and Lars and sorry for all the trouble around this.
          Hide
          Adrian Cole added a comment -

          opening to add another patch

          warn and convert the following providers. note that this patch should be reverted after 1 release:

          ec2 -> aws-ec2
          cloudservers -> cloudservers-us

          Show
          Adrian Cole added a comment - opening to add another patch warn and convert the following providers. note that this patch should be reverted after 1 release: ec2 -> aws-ec2 cloudservers -> cloudservers-us
          Hide
          Adrian Cole added a comment -

          note this is the warning message one will get when using the old provider keys:

          2011-02-07 16:27:02,426 WARN [org.apache.whirr.service.ComputeServiceContextBuilder] (main) please use provider "aws-ec2" instead of "ec2"

          Show
          Adrian Cole added a comment - note this is the warning message one will get when using the old provider keys: 2011-02-07 16:27:02,426 WARN [org.apache.whirr.service.ComputeServiceContextBuilder] (main) please use provider "aws-ec2" instead of "ec2"
          Hide
          Adrian Cole added a comment -

          this patch warns before converting ec2->aws-ec2 and cloudservers -> clouservers-us

          tested with cassandra on both ec2 and cloudservers

          Show
          Adrian Cole added a comment - this patch warns before converting ec2->aws-ec2 and cloudservers -> clouservers-us tested with cassandra on both ec2 and cloudservers
          Hide
          Andrei Savu added a comment -

          +1 I've just committed the new patch. Thanks Adrian!

          Show
          Andrei Savu added a comment - +1 I've just committed the new patch. Thanks Adrian!

            People

            • Assignee:
              Adrian Cole
              Reporter:
              Tom White
            • Votes:
              1 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development