[GORA-120] Dirty fields are not correctly applied after serialization and map clearance

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.2
• Assignee: Unassigned
• Reporter: Ferdy
• Created: 18/Apr/12 11:51
• Updated: 01/Jul/13 05:05
• Resolved: 19/Apr/12 15:11

Description

Dirty fields are not correctly applied in these situations:

-All entries in the map are incorrectly made dirty when a Persistent is deserialized.
-Also, due to a separate bug all non-map entries that are deserialized are made dirty too. (This is pure coincidence as the bugs have different causes).
-Finally, when a map is cleared but some entries that are put afterwards are actually the same as before, it should be noticed as non-dirty. (This happens in Nutchgora a lot: getInlinks.clear(), getOutlinks.clear(), getHeaders.clear() etc.).

Because of this a lot of fields are dirty when they in fact are not. It is never the other way around. Therefore, this bug purely affects performance.

Patch will follow shortly. Let me know if you have any comments.

Issue Links

• incorporates: GORA-33 Read map entries are all DIRTY by default
• is depended upon by: NUTCH-1340 Increase scalability by only removing markers when they actually exist for DbUpdaterReducer