Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-79

Fault tolerant searching.

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Won't Fix
    • None
    • None
    • None
    • None

    Description

      I have finally managed to prepare first version of fault tolerant searching I have promised long time ago.
      It reads server configuration from search-groups.txt file (in startup directory or directory specified by searcher.dir) if no search-servers.txt file is present. If search-servers.txt is presentit would be read and handled as previously.
      ---------------------------------------------------
      Format of search-groups.txt:

      • <pre>
      • search.group.count=[int]
      • search.group.name.[i]=[string] (for i=0 to count-1)
      • For each name:
      • [name].part.count=[int] partitionCount
      • [name].part.[i].host=[string] (for i=0 to partitionCount-1)
      • [name].part.[i].port=int (for i=0 to partitionCount-1)
      • Example:
      • search.group.count=2
      • search.group.name.0=master
      • search.group.name.1=backup
      • master.part.count=2
      • master.part.0.host=host1
      • master.part.0.port=7777
      • master.part.1.host=host2
      • master.part.1.port=7777
      • backup.part.count=2
      • backup.part.0.host=host3
      • backup.part.0.port=7777
      • backup.part.1.host=host4
      • backup.part.1.port=7777
      • </pre>.
        ------------------------------------------------

      If more than one search group is defined in configuration file requests are distributed among groups in round-robin fashion. If one of the servers from the group fails to respond the whole group is treated as inactive and removed from the pool used to distributed requests. There is a separate recovery thread that every "searcher.recovery.delay" seconds (default 60) tries to check if inactive became alive and if so adds it back to the pool of active groups.

      Attachments

        1. patch
          65 kB
          Piotr Kosiorowski

        Activity

          People

            Unassigned Unassigned
            pkosiorowski Piotr Kosiorowski
            Votes:
            1 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: