Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-1747

Use AtomicInteger as semaphore in Fetcher

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.8
    • Fix Version/s: 1.9
    • Component/s: fetcher
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      In Fetcher we currently use

      Set<FetchItem> inProgress = Collections.synchronizedSet(new HashSet<FetchItem>());

      as semaphore within the FetchItemQueues to keep track of the URLs being fetched and prevent threads from pulling from them. It works fine but we could use AtomicIntegers instead as all we need is the counts, not the contents.

      This change would have little impact on the performance but would make the code a bit cleaner.

        Attachments

        1. NUTCH-1747-trunk.patch
          2 kB
          Julien Nioche

          Activity

            People

            • Assignee:
              jnioche Julien Nioche
              Reporter:
              jnioche Julien Nioche
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: