[NUTCH-1352] Improve regex urlfilters/normalizers synchronization

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: nutchgora, 1.6
• Patch Info: Patch Available
• Assignee: Unassigned
• Reporter: Ferdy
• Created: 07/May/12 08:55
• Updated: 22/May/13 03:53
• Resolved: 12/Jun/12 10:15

Description

I noticed that during fetching a lot of the time the fetcherthreads are blocking on a monitor because of outlink normalizing/filtering. The cause of this: Some of the regex plugins use single lock synchronization.

This patch improves throughput by removing synchronization locks and replace them with threadlocals were needed.

It has been extensively tested in production. I will commit this later today when no objection.