[ACCUMULO-3983] Create tablet locator test suite

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Keith Turner
• Created: 31/Aug/15 17:14
• Updated: 11/Jun/19 06:06

Description

Looking at ACCUMULO-3982 made me take a look at the tablet locator source code. When I was looking at the code, I thought it would be nice if there were more parallelism in doing metadata lookups (there is a write lock for updating the cache). However this can be tricky because if 100 threads need the same info thats not in the cache we do not want to lookup that information 100 times in metadata. This made me think we probably need a test for the locator that measures certain things under different circumstances.

For example we probably want to measure the following things :

• Total cache memory usage
• Number of lookups made to the metadata table
• Response time of cache

Would want to run at least the following scenarios when measuring the above.

• N threads lookup N different tablet locations against empty cache and populate cache
• N threads lookup 1 tablet location against empty cache and populated cache
• N threads invalidate N key extents and request location
• N threads invalidate 1 key extents and request location

More scenarios would be needed.

Ideally an implementation of tablet locator would minimize the time it takes a thread to get a response and it would minimize the number of metadata table request. Hopefully having this test suite could drive better implementations. Need to be careful that the test scenarios are realistic.

Would probably want to make the test suite capable of running on multiple machines in-order to properly simulate sharing the metadata table.

Issue Links

• relates to: ACCUMULO-3982 Add LRU mechanism to address memory concerns and possibly pre-fetch Tablet Locations.