Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-1448

Redirected urls should be handled more cleanly (more like an outlink url)

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.1
    • None
    • None

    Description

      This is specifically for Nutch2.x. Handling a redirects url like an outlink is much more cleaner because this makes it more simple to trace how new urls are added to the webpage database. Instant fetching of redirects won't work, but this is a small price to pay. (Note that this currently does not work at all, because the http.max.redirect property has no effect). Will be attaching a patch in the upcoming days.

      Attachments

        1. nutch-1448.txt
          22 kB
          Ferdy

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. NUTCH-1461 Problem with TableUtil

          • Major - Major loss of function.
          • Closed

          Activity

            People

              Unassigned Unassigned
              ferdy.g Ferdy
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: