[NUTCH-719] fetchQueues.totalSize incorrect in Fetcher2

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.1
• Component/s: fetcher
• Assignee: Julien Nioche
• Reporter: Julien Nioche
• Created: 12/Mar/09 14:41
• Updated: 23/Feb/10 08:49
• Resolved: 19/Feb/10 18:51

Description

I had a look at the logs generated by Fetcher2 and found cases where there were no active fetchQueues but fetchQueues.totalSize was != 0

fetcher.Fetcher2 - -activeThreads=200, spinWaiting=200, fetchQueues.totalSize=1, fetchQueues=0

since the code relies on fetchQueues.totalSize to determine whether the work is finished or not the task is blocked until the abortion mechanism kicks in

2009-03-12 09:27:38,977 WARN fetcher.Fetcher2 - Aborting with 200 hung threads.

could that be a synchronisation issue? any ideas?