[HBASE-14028] DistributedLogReplay drops edits when ITBLL 125M

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Later
• Affects Version/s: 1.2.0
• Component/s: Recovery
• Assignee: Unassigned
• Reporter: Michael Stack
• Created: 06/Jul/15 23:01
• Updated: 16/Jun/22 16:45
• Resolved: 22/Sep/15 03:56

Description

Testing DLR before 1.2.0RC gets cut, we are dropping edits.

Issue seems to be around replay into a deployed region that is on a server that dies before all edits have finished replaying. Logging is sparse on sequenceid accounting so can't tell for sure how it is happening (and if our now accounting by Store is messing up DLR). Digging.

I notice also that DLR does not refresh its cache of region location on error – it just keeps trying till whole WAL fails.... 8 retries...about 30 seconds. We could do a bit of refactor and have the replay find region in new location if moved during DLR replay.

Issue Links

• is depended upon by: HBASE-7006 [MTTR] Improve Region Server Recovery Time - Distributed Log Replay