Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-1724

Real Basic Core Management with Zookeeper

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.4
    • Fix Version/s: 4.9, 6.0
    • Component/s: multicore
    • Labels:
      None

      Description

      Though we're implementing cloud, I need something real soon I can
      play with and deploy. So this'll be a patch that only deploys
      new cores, and that's about it. The arch is real simple:

      On Zookeeper there'll be a directory that contains files that
      represent the state of the cores of a given set of servers which
      will look like the following:

      /production/cores-1.txt
      /production/cores-2.txt
      /production/core-host-1-actual.txt (ephemeral node per host)

      Where each core-N.txt file contains:

      hostname,corename,instanceDir,coredownloadpath

      coredownloadpath is a URL such as file://, http://, hftp://, hdfs://, ftp://, etc

      and

      core-host-actual.txt contains:

      hostname,corename,instanceDir,size

      Everytime a new core-N.txt file is added, the listening host
      finds it's entry in the list and begins the process of trying to
      match the entries. Upon completion, it updates it's
      /core-host-1-actual.txt file to it's completed state or logs an error.

      When all host actual files are written (without errors), then a
      new core-1-actual.txt file is written which can be picked up by
      another process that can create a new core proxy.

        Attachments

        1. SOLR-1724.patch
          24 kB
          Jason Rutherglen
        2. SOLR-1724.patch
          49 kB
          Jason Rutherglen
        3. SOLR-1724.patch
          54 kB
          Jason Rutherglen
        4. SOLR-1724.patch
          58 kB
          Jason Rutherglen
        5. SOLR-1724.patch
          58 kB
          Jason Rutherglen
        6. SOLR-1724.patch
          61 kB
          Jason Rutherglen
        7. SOLR-1724.patch
          64 kB
          Jason Rutherglen
        8. SOLR-1724.patch
          99 kB
          Jason Rutherglen
        9. SOLR-1724.patch
          83 kB
          Jason Rutherglen
        10. SOLR-1724.patch
          85 kB
          Jason Rutherglen
        11. SOLR-1724.patch
          114 kB
          Jason Rutherglen
        12. hadoop-0.20.2-dev-test.jar
          1.49 MB
          Jason Rutherglen
        13. hadoop-0.20.2-dev-core.jar
          2.56 MB
          Jason Rutherglen
        14. gson-1.4.jar
          162 kB
          Jason Rutherglen
        15. commons-lang-2.4.jar
          256 kB
          Jason Rutherglen

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jasonrutherglen Jason Rutherglen
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: