Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-1074

topN is ignored with maxNumSegments

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.3
    • 1.4
    • generator
    • None

    Description

      When generating segments with topN and maxNumSegments, topN is not respected. It looks like the first generated segment contains topN * maxNumSegments of URLs's, at least the number of map input records roughly matches.

      Attachments

        1. generator_fix.patch
          2 kB
          Robert Thomson

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. NUTCH-762 Alternative Generator which can generate several segments in one parse of the crawlDB

          • Major - Major loss of function.
          • Closed

          Activity

            People

              markus17 Markus Jelsma
              markus17 Markus Jelsma
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: