Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1564

Metadata for dead Oozie servers should remain in ZK

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: trunk
    • Fix Version/s: None
    • Component/s: HA
    • Labels:
      None

      Description

      As per the discussion in OOZIE-1561, we should have two services in the service discovery on ZK:

      1. Contains the metadata for any Oozie server that has ever connected, even ones that are currently dead
        • This would be used by log streaming so that dead servers are always identified
        • I think there is an easy way to do this with a setting in Curator, but I have to test it. If not, then we'll have to manually create/manage the ZNodes.
      2. Contains the metadata for any Oozie server that is still alive (the current behavior)
        • This would be used by the mod calculation so only alive servers are included

      Maintaining two lists will avoid the need to do any pinging or tricky heartbeating.

      We'd have to add a new admin command to cleanup the dead servers. It would look at which servers are not in the alive list and remove those.

      It would also be good to update the admin command that lists servers to specify which ones are alive and which are dead; and possibly have filtering (i.e. show only alive servers, show only dead servers, show all servers)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                bowenzhangusa Bowen Zhang
                Reporter:
                rkanter Robert Kanter
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: