[NUTCH-1341] NotModified time set to now but page not modified

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5
• Fix Version/s: 1.6
• Patch Info: Patch Available
• Assignee: Markus Jelsma
• Reporter: Markus Jelsma
• Created: 19/Apr/12 14:50
• Updated: 22/May/13 03:53
• Resolved: 23/Oct/12 13:29

Description

Servers tend to respond with incorrect or no value for LastModified. By comparing signatures or when (fetch.getStatus() == CrawlDatum.STATUS_FETCH_NOTMODIFIED) the reducer correctly sets the db_notmodified status for the CrawlDatum. The modifiedTime value, however, is not set accordingly.