[ACCUMULO-1152] Add more sanity checks to limit the damage of multiple assignment

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Assignee: Keith Turner
• Reporter: Keith Turner
• Created: 05/Mar/13 19:49
• Updated: 24/Sep/13 22:05
• Resolved: 24/Sep/13 22:05

Description

When an issue like ACCUMULO-954 comes along and causes a tablet to be hosted on multiple tablet servers, its nice to have sanity checks that limit the damage this can cause.

Accumulo already has a sanity check on writes to the metadata table that ensures the tablet server making the write still holds a lock. This has reliably triggered in cases of multiple assignment bugs. It would be nice to have more checks like this. Below are some places I think checks would be useful, are there more?

• Tserver attempts to positively check it holds its lock before getting a new walog.
• Clients take some action to clear lockless tservers from their metadata table cache. This would help prevent writing data to a zombie tserver that may lose data, or reading stale data from a zombie tserver.

Issue Links

• is depended upon by: ACCUMULO-1000 support compare and set
• relates to: ACCUMULO-1165 Use a unique SystemToken (credentials) for each server instance (TServer, etc.)