[NUTCH-269] CrawlDbReducer: OOME because no upper-bound on inlinks count - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Trivial
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 1.1
Component/s: None
Labels:
None

Description

A CrawlDB update repeatedly OOME'd because an URL had hundreds of thousands of inlinks (The british foriegn office likes putting a clear.gif multiple times into each page: http://www.fco.gov.uk/Xcelerate/graphics/images/fcomain/clear.gif).

Attachments

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

too-many-links.patch
16/May/06 05:30
1 kB
Michael Stack
too-many-links2.patch
16/May/06 05:45
2 kB
Michael Stack

Activity

People

Assignee:: Julien Nioche

Reporter:: Michael Stack

Votes:: 0 Vote for this issue

Watchers:: 0 Start watching this issue

Dates

Created:: 16/May/06 05:29

Updated:: 22/May/13 03:54

Resolved:: 08/Jan/10 12:02