[CONNECTORS-243] Web crawler must get the "Last-Modified" HTTP header and pass it as metadata to output

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: ManifoldCF 0.2
• Fix Version/s: ManifoldCF 0.3
• Component/s: Web connector
• Labels: last-modified
• Assignee: Karl Wright
• Reporter: Jan Høydahl
• Created: 22/Aug/11 15:09
• Updated: 22/Aug/11 23:58
• Resolved: 22/Aug/11 23:58

Description

Last-Modified is important in web search, at it may be used for (de)boosting based on date.
In fact, ManifoldCF should have the ability to parse any (or all) HTTP headers from source document and pass it on.