Whirr
  1. Whirr
  2. WHIRR-150

Allow retrieval of instance roles

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.3.0
    • Component/s: core
    • Labels:
      None

      Description

      After a cluster has been launched there is no way to access the roles running on particular instances. It would be useful to find the workers in a Hadoop cluster, so you could operate on them (e.g. restarting the daemons on them).

      How this is implemented is probably different for different clouds. E.g. on EC2 it could be accomplished using tags, on other platforms a local file might be sufficient, and a blobstore might be another way.

      1. WHIRR-150.patch
        19 kB
        Tom White
      2. WHIRR-150.patch
        2 kB
        Tom White

        Issue Links

          Activity

          Tom White created issue -
          Hide
          Tom White added a comment -

          Here's a simple approach, which just writes information about the instances in the cluster to a tab-delimited plain text file called ~/.whirr/<cluster>/instances. It should be easy to use unix tools to parse this file and operate on instances in the cluster. Here's an example of the file for a Hadoop cluster:

          us-east-1/i-54cbe839     tt,dn     ec2-67-202-4-185.compute-1.amazonaws.com     10.240.61.238
          us-east-1/i-0ecbe863     jt,nn     ec2-72-44-46-97.compute-1.amazonaws.com     10.240.62.31
          

          The columns represent ID, roles, public address, private address.

          Show
          Tom White added a comment - Here's a simple approach, which just writes information about the instances in the cluster to a tab-delimited plain text file called ~/.whirr/<cluster>/instances . It should be easy to use unix tools to parse this file and operate on instances in the cluster. Here's an example of the file for a Hadoop cluster: us-east-1/i-54cbe839 tt,dn ec2-67-202-4-185.compute-1.amazonaws.com 10.240.61.238 us-east-1/i-0ecbe863 jt,nn ec2-72-44-46-97.compute-1.amazonaws.com 10.240.62.31 The columns represent ID, roles, public address, private address.
          Tom White made changes -
          Field Original Value New Value
          Attachment WHIRR-150.patch [ 12465874 ]
          Tom White made changes -
          Assignee Tom White [ tomwhite ]
          Fix Version/s 0.3.0 [ 12315487 ]
          Tom White made changes -
          Link This issue blocks WHIRR-56 [ WHIRR-56 ]
          Tom White made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Lars George added a comment -

          +1

          This is sweet and easily human readable as well as parsable by scripts. Thanks!

          Show
          Lars George added a comment - +1 This is sweet and easily human readable as well as parsable by scripts. Thanks!
          Hide
          Andrei Savu added a comment -

          This file could be useful for all current services and future ones. Why only for Hadoop? Is there any particular reason why this file is not generated in the LaunchClusterCommand class after the cluster is started?

          Unrelated: I also think that HadoopNameNodeClusterActionHandler.getConfigDir should be a method in ClusterSpec.

          Show
          Andrei Savu added a comment - This file could be useful for all current services and future ones. Why only for Hadoop? Is there any particular reason why this file is not generated in the LaunchClusterCommand class after the cluster is started? Unrelated: I also think that HadoopNameNodeClusterActionHandler.getConfigDir should be a method in ClusterSpec .
          Hide
          Tom White added a comment -

          +1 to moving this to LaunchClusterCommand and making the local cluster directory a common concept. I'll produce a new patch for this.

          Show
          Tom White added a comment - +1 to moving this to LaunchClusterCommand and making the local cluster directory a common concept. I'll produce a new patch for this.
          Hide
          Tom White added a comment -

          Here's a patch which writes the instances file for all clusters. This is done by Service, not LaunchClusterCommand, since it is not particularly CLI-specific.

          I had to move DnsUtil to core since its a dependency, so I moved DnsUtilTest to core's integration directory (part of WHIRR-162).

          Show
          Tom White added a comment - Here's a patch which writes the instances file for all clusters. This is done by Service, not LaunchClusterCommand, since it is not particularly CLI-specific. I had to move DnsUtil to core since its a dependency, so I moved DnsUtilTest to core's integration directory (part of WHIRR-162 ).
          Tom White made changes -
          Attachment WHIRR-150.patch [ 12466369 ]
          Hide
          Andrei Savu added a comment -

          (+1) Looks good. All tests are passing for me.

          Show
          Andrei Savu added a comment - (+1) Looks good. All tests are passing for me.
          Hide
          Tom White added a comment -

          I've just committed this. Thanks for the review, Andrei.

          Show
          Tom White added a comment - I've just committed this. Thanks for the review, Andrei.
          Tom White made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          8d 11h 8m 1 Tom White 09/Dec/10 17:29
          Patch Available Patch Available Resolved Resolved
          7d 3h 55m 1 Tom White 16/Dec/10 21:25

            People

            • Assignee:
              Tom White
              Reporter:
              Tom White
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development