Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-1504

Pluggable url partitioner

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.6
    • Fix Version/s: None
    • Component/s: generator
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      At present, the url partition logic is hard wired inside nutch core. It should be pluggable like FetchSchedule customized via nutch-site.xml.

      There might be use cases where a single domain needs to be partioned on some custom logic. The existing UrlPartitioner cannot handle such cases.

      Hence the requirement.

        Attachments

        1. custom.partitioner.patch
          6 kB
          Sourajit Basak

          Activity

            People

            • Assignee:
              lewismc Lewis John McGibbney
              Reporter:
              sourajit Sourajit Basak
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: