[HADOOP-6332] Large-scale Automated Test Framework

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.21.0
• Fix Version/s: 0.21.0
• Component/s: test
• Hadoop Flags: Reviewed
• Tags: herriot
• Assignee: Konstantin I Boudnik
• Reporter: Arun Murthy
• Created: 25/Oct/09 06:29
• Updated: 02/May/13 02:29
• Resolved: 25/May/10 01:16

Description

Hadoop would benefit from having a large-scale, automated, test-framework. This jira is meant to be a master-jira to track relevant work.

---

The proposal is a junit-based, large-scale test framework which would run against real clusters.

There are several pieces we need to achieve this goal:

1. A set of utilities we can use in junit-based tests to work with real, large-scale hadoop clusters. E.g. utilities to bring up to deploy, start & stop clusters, bring down tasktrackers, datanodes, entire racks of both etc.
2. Enhanced control-ability and inspect-ability of the various components in the system e.g. daemons such as namenode, jobtracker should expose their data-structures for query/manipulation etc. Tests would be much more relevant if we could for e.g. query for specific states of the jobtracker, scheduler etc. Clearly these apis should not be part of the production clusters - hence the proposal is to use aspectj to weave these new apis to debug-deployments.

---

Related note: we should break up our tests into at least 3 categories:

1. src/test/unit -> Real unit tests using mock objects (e.g. HDFS-669 & MAPREDUCE-1050).
2. src/test/integration -> Current junit tests with Mini* clusters etc.
3. src/test/system -> HADOOP-6332 and it's children

Issue Links

• blocks: HADOOP-6752 Remote cluster control functionality needs JavaDocs improvement
• incorporates: MAPREDUCE-1154 Large-scale, automated test framwork for Map-Reduce
• is depended upon by: HADOOP-6788 [Herriot] Exception exclusion functionality is not working correctly.
• is depended upon by: HADOOP-6819 [Herriot] Shell command for getting the new exceptions in the logs returning exitcode 1 after executing successfully.
• is depended upon by: HADOOP-6869 Functionality to create file or folder on a remote daemon side
• is depended upon by: HADOOP-6777 Implement a functionality for suspend and resume a process.
• is depended upon by: HADOOP-6836 [Herriot]: Generic method for adding/modifying the attributes for new configuration.
• is depended upon by: HADOOP-6772 Utilities for system tests specific.
• is related to: HADOOP-6789 set-version should precede Herriot artifacts installation target
• is related to: HADOOP-6790 Instrumented (Herriot) build uses too wide mask to include aspect files.
• is related to: HADOOP-1257 a distributed junit test runner
• is related to: HDFS-708 A stress-test tool for HDFS.
• is related to: HADOOP-6786 test-patch needs to verify Herriot integrity
• is required by: MAPREDUCE-1653 Add apache header to UserNamePermission.java
• is required by: MAPREDUCE-1655 Automate the local file permission checking when a job is running.
• relates to: HADOOP-6828 Herrior uses old way of accessing logs directories
• relates to: HADOOP-6248 Circus: Proposal and Preliminary Code for a Hadoop System Testing Framework
• relates to: MAPREDUCE-1774 Large-scale Automated Framework
• relates to: HDFS-1341 Provide a mechanism to run functional tests on a real cluster
• relates to: HDFS-1134 Large-scale Automated Framework
• relates to: HADOOP-6784 Create Wiki document about Herriot Test Framework and test development guide