Nutch
  1. Nutch
  2. NUTCH-1356

ParseUtil use ExecutorService instead of manually thread handling.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: nutchgora, 1.6
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      Because ParseUtil manages it's own parser threads by creating a thread for every parse it sometimes happens that specific parsers are very expensive. For example, parsers that have threadlocal fields will initialize them for every item to be parsed.

      By simply introducing a caching ExecutorService the ParseUtil will be able to cache threads therefore parsing more efficient. See attached patch.

      1. NUTCH-1356.patch
        4 kB
        Ferdy Galema
      2. NUTCH-1356-trunk.patch
        3 kB
        Ferdy Galema
      3. NUTCH-1356-trunk-v2.patch
        3 kB
        Ferdy Galema

        Activity

          People

          • Assignee:
            Unassigned
            Reporter:
            Ferdy Galema
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development