Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-426

parse-js skips parsing if found URL fails java.net.URL parse

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.9.0
    • 0.9.0
    • fetcher
    • None

    Description

      The parse-js plugin in getJSLinks tries a regex looking for likely URLs against a string of javascript. Any matches that do not begin 'www' are given to java.net.URL with base URL to test 'URLness'. Often this test will fail. Currently, when it fails, nutch skips processing any more of the javascript snippet logs an error.

      Attachments

        1. nutch426.patch
          1 kB
          Michael Stack

        Activity

          People

            ab Andrzej Bialecki
            stack Michael Stack
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: