Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-1531

URL filtering takes long time for very long URLs

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Duplicate
    • 1.6, 2.1, 1.7, 2.2
    • None
    • None
    • None
    • Patch Available

    Description

      Filtering very long urls (such as base64 image generators) take long time (hours). On reducing phase it locks down all the system for hours. Therefore some URL length limitation needed. We attached a little patch for this improvement.

      Attachments

        1. max_url_length.diff
          2 kB
          Fırat KÜÇÜK
        2. test_case.txt
          378 kB
          Fırat KÜÇÜK

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. NUTCH-1314 Impose a limit on the length of outlink target urls

          • Major - Major loss of function.
          • Closed

          Activity

            People

              Unassigned Unassigned
              firatkucuk Fırat KÜÇÜK
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: