Whirr
  1. Whirr
  2. WHIRR-414

whirr can have a non-zero return code and unterminated (orphaned) host instances

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.7.0
    • Component/s: core
    • Labels:
      None
    • Environment:

      EC2, commandline whirr

      Description

      Whirr can fail to completely start a cluster and indicates this with a non-zero return code. In many (currently intermittent) partial failure scenarios, there are resources still active (EC2 machine instances, in my experience) that are not cleaned up.

      The log contains "IOException: Too many instance failed while bootstrapping!" when I have seen orphaned nodes.

      A non-zero return code should guarantee that all resources are cleaned up. Without this post-condition, these failures require manual inspection and cleanup to stop useless expenses (which is why I marked this bug critical; it needs to be addressed for any kind of cron job triggered whirr).

      1. WHIRR-414-ignore-missing-instances-file.patch
        3 kB
        Andrei Savu
      2. WHIRR-414-ignore-missing-instances-file.patch
        3 kB
        Andrei Savu
      3. WHIRR-414.patch
        6 kB
        Andrei Savu
      4. WHIRR-414.patch
        8 kB
        Andrei Savu
      5. WHIRR-414.patch
        7 kB
        David Alves
      6. WHIRR-414.patch
        7 kB
        David Alves
      7. WHIRR-414.patch
        2 kB
        Andrei Savu

        Activity

        Andrei Savu made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Andrei Savu made changes -
        Andrei Savu made changes -
        Status Reopened [ 4 ] Patch Available [ 10002 ]
        Andrei Savu made changes -
        Andrei Savu made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Andrei Savu made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Andrei Savu made changes -
        Attachment WHIRR-414.patch [ 12503190 ]
        Andrei Savu made changes -
        Attachment WHIRR-414.patch [ 12503003 ]
        David Alves made changes -
        Assignee David Alves [ dr-alves ] Andrei Savu [ savu.andrei ]
        David Alves made changes -
        Attachment WHIRR-414.patch [ 12502969 ]
        David Alves made changes -
        Assignee Andrei Savu [ savu.andrei ] David Alves [ dr-alves ]
        David Alves made changes -
        Attachment WHIRR-414.patch [ 12502965 ]
        Andrei Savu made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Assignee Andrei Savu [ savu.andrei ]
        Fix Version/s 0.7.0 [ 12317571 ]
        Andrei Savu made changes -
        Field Original Value New Value
        Attachment WHIRR-414.patch [ 12501262 ]
        Paul Baclace created issue -

          People

          • Assignee:
            Andrei Savu
            Reporter:
            Paul Baclace
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development