[HBASE-2341] Suite of test scripts that a.) load a cluster with a verifiable dataset and b.) do random kills of regionserver+datanodes in small cluster

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Labels: moved_from_0_20_5
• Assignee: Unassigned
• Reporter: Michael Stack
• Created: 17/Mar/10 23:25
• Updated: 11/Jun/22 23:22
• Resolved: 16/Jul/14 20:45

Description

We just filed hbase-2340 but discussion up on irc has it that we need something more hardcore than pussy-footing inside a single jvm as hdfs-2340 does. The point was made (tlipcon) that its hard to ensure real recovery working if all is in the one JVM.

So, this issue is about scripts that can:

+ load a cluster with a dataset that we can 'verify' as in we can tell if it has holes in it, if data has been lost.
+ script that does random kill of a random node on some random occasion
+ Script that can check cluster for data loss

All above should work while cluster is under load.

The above would not sit under junit.

This looks like a suite that we'd want to run up in ec2 using Andrew's scripts and our donated aws credits.

16:12 < tlipcon> here's my goal: we have a 5 node cluster in the back room. I want to run hbase on that at near full load for a week straight while some process goes around screwing with it
16:12 < tlipcon> then I want to verify that I didn't lose a single edit over that week

Issue Links

• is related to: HBASE-2343 [EC2] Test harness