Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-1356

ParseUtil use ExecutorService instead of manually thread handling.

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: nutchgora, 1.6
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      Because ParseUtil manages it's own parser threads by creating a thread for every parse it sometimes happens that specific parsers are very expensive. For example, parsers that have threadlocal fields will initialize them for every item to be parsed.

      By simply introducing a caching ExecutorService the ParseUtil will be able to cache threads therefore parsing more efficient. See attached patch.

        Attachments

        1. NUTCH-1356.patch
          4 kB
          Ferdy Galema
        2. NUTCH-1356-trunk.patch
          3 kB
          Ferdy Galema
        3. NUTCH-1356-trunk-v2.patch
          3 kB
          Ferdy Galema

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ferdy.g Ferdy Galema
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: