Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-1747

Use AtomicInteger as semaphore in Fetcher

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 1.8
    • 1.9
    • fetcher
    • None
    • Patch Available

    Description

      In Fetcher we currently use

      Set<FetchItem> inProgress = Collections.synchronizedSet(new HashSet<FetchItem>());

      as semaphore within the FetchItemQueues to keep track of the URLs being fetched and prevent threads from pulling from them. It works fine but we could use AtomicIntegers instead as all we need is the counts, not the contents.

      This change would have little impact on the performance but would make the code a bit cleaner.

      Attachments

        1. NUTCH-1747-trunk.patch
          2 kB
          Julien Nioche

        Activity

          People

            jnioche Julien Nioche
            jnioche Julien Nioche
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: