Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
1.4
-
None
-
None
Description
Though we're implementing cloud, I need something real soon I can
play with and deploy. So this'll be a patch that only deploys
new cores, and that's about it. The arch is real simple:
On Zookeeper there'll be a directory that contains files that
represent the state of the cores of a given set of servers which
will look like the following:
/production/cores-1.txt
/production/cores-2.txt
/production/core-host-1-actual.txt (ephemeral node per host)
Where each core-N.txt file contains:
hostname,corename,instanceDir,coredownloadpath
coredownloadpath is a URL such as file://, http://, hftp://, hdfs://, ftp://, etc
and
core-host-actual.txt contains:
hostname,corename,instanceDir,size
Everytime a new core-N.txt file is added, the listening host
finds it's entry in the list and begins the process of trying to
match the entries. Upon completion, it updates it's
/core-host-1-actual.txt file to it's completed state or logs an error.
When all host actual files are written (without errors), then a
new core-1-actual.txt file is written which can be picked up by
another process that can create a new core proxy.