Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-426

parse-js skips parsing if found URL fails java.net.URL parse

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.9.0
    • Component/s: fetcher
    • Labels:
      None

      Description

      The parse-js plugin in getJSLinks tries a regex looking for likely URLs against a string of javascript. Any matches that do not begin 'www' are given to java.net.URL with base URL to test 'URLness'. Often this test will fail. Currently, when it fails, nutch skips processing any more of the javascript snippet logs an error.

        Attachments

        1. nutch426.patch
          1 kB
          stack

          Activity

            People

            • Assignee:
              ab Andrzej Bialecki
              Reporter:
              stack stack
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: