Whirr
  1. Whirr
  2. WHIRR-150

Allow retrieval of instance roles

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.3.0
    • Component/s: core
    • Labels:
      None

      Description

      After a cluster has been launched there is no way to access the roles running on particular instances. It would be useful to find the workers in a Hadoop cluster, so you could operate on them (e.g. restarting the daemons on them).

      How this is implemented is probably different for different clouds. E.g. on EC2 it could be accomplished using tags, on other platforms a local file might be sufficient, and a blobstore might be another way.

      1. WHIRR-150.patch
        19 kB
        Tom White
      2. WHIRR-150.patch
        2 kB
        Tom White

        Issue Links

          Activity

          Tom White made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Tom White added a comment -

          I've just committed this. Thanks for the review, Andrei.

          Show
          Tom White added a comment - I've just committed this. Thanks for the review, Andrei.
          Hide
          Andrei Savu added a comment -

          (+1) Looks good. All tests are passing for me.

          Show
          Andrei Savu added a comment - (+1) Looks good. All tests are passing for me.
          Tom White made changes -
          Attachment WHIRR-150.patch [ 12466369 ]
          Hide
          Tom White added a comment -

          Here's a patch which writes the instances file for all clusters. This is done by Service, not LaunchClusterCommand, since it is not particularly CLI-specific.

          I had to move DnsUtil to core since its a dependency, so I moved DnsUtilTest to core's integration directory (part of WHIRR-162).

          Show
          Tom White added a comment - Here's a patch which writes the instances file for all clusters. This is done by Service, not LaunchClusterCommand, since it is not particularly CLI-specific. I had to move DnsUtil to core since its a dependency, so I moved DnsUtilTest to core's integration directory (part of WHIRR-162 ).
          Hide
          Tom White added a comment -

          +1 to moving this to LaunchClusterCommand and making the local cluster directory a common concept. I'll produce a new patch for this.

          Show
          Tom White added a comment - +1 to moving this to LaunchClusterCommand and making the local cluster directory a common concept. I'll produce a new patch for this.
          Hide
          Andrei Savu added a comment -

          This file could be useful for all current services and future ones. Why only for Hadoop? Is there any particular reason why this file is not generated in the LaunchClusterCommand class after the cluster is started?

          Unrelated: I also think that HadoopNameNodeClusterActionHandler.getConfigDir should be a method in ClusterSpec.

          Show
          Andrei Savu added a comment - This file could be useful for all current services and future ones. Why only for Hadoop? Is there any particular reason why this file is not generated in the LaunchClusterCommand class after the cluster is started? Unrelated: I also think that HadoopNameNodeClusterActionHandler.getConfigDir should be a method in ClusterSpec .
          Hide
          Lars George added a comment -

          +1

          This is sweet and easily human readable as well as parsable by scripts. Thanks!

          Show
          Lars George added a comment - +1 This is sweet and easily human readable as well as parsable by scripts. Thanks!
          Tom White made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Tom White made changes -
          Link This issue blocks WHIRR-56 [ WHIRR-56 ]
          Tom White made changes -
          Assignee Tom White [ tomwhite ]
          Fix Version/s 0.3.0 [ 12315487 ]
          Tom White made changes -
          Field Original Value New Value
          Attachment WHIRR-150.patch [ 12465874 ]
          Hide
          Tom White added a comment -

          Here's a simple approach, which just writes information about the instances in the cluster to a tab-delimited plain text file called ~/.whirr/<cluster>/instances. It should be easy to use unix tools to parse this file and operate on instances in the cluster. Here's an example of the file for a Hadoop cluster:

          us-east-1/i-54cbe839     tt,dn     ec2-67-202-4-185.compute-1.amazonaws.com     10.240.61.238
          us-east-1/i-0ecbe863     jt,nn     ec2-72-44-46-97.compute-1.amazonaws.com     10.240.62.31
          

          The columns represent ID, roles, public address, private address.

          Show
          Tom White added a comment - Here's a simple approach, which just writes information about the instances in the cluster to a tab-delimited plain text file called ~/.whirr/<cluster>/instances . It should be easy to use unix tools to parse this file and operate on instances in the cluster. Here's an example of the file for a Hadoop cluster: us-east-1/i-54cbe839 tt,dn ec2-67-202-4-185.compute-1.amazonaws.com 10.240.61.238 us-east-1/i-0ecbe863 jt,nn ec2-72-44-46-97.compute-1.amazonaws.com 10.240.62.31 The columns represent ID, roles, public address, private address.
          Tom White created issue -

            People

            • Assignee:
              Tom White
              Reporter:
              Tom White
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development