[ACCUMULO-4124] integration tests that use map-reduce run under the yarn instance when using IntegrationTestMapReduce

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.8.0
• Component/s: test

Environment

running integration tests as a map-reduce job on a small cluster

• Assignee: Eric C. Newton
• Reporter: Eric C. Newton
• Created: 02/Feb/16 17:21
• Updated: 02/Feb/16 20:27
• Resolved: 02/Feb/16 20:27

Description

While attempting to run the Integration Tests under a map-reduce job using the fat mrit jar, I found that all the InputFormat and OutputFormat tests were failing. Checking on the cluster, I saw that the ITs were kicking off new applications within the yarn cluster.

There are several problems with this:

1) the cluster is busy running ITs, so it doesn't have resources to be the M/R infrastructure for a test
2) zookeeper is consistently started on localhost, and the sub-map jobs cannot reach the zookeeper on the host that started it
3) files prepared by the test do not exist in order for the test to run because it is running on another host

This contributes to 15 of the 192 tests throwing unexpected exceptions. That is, they do not run to completion, either with test success, or test failure. They also create a long tail of timeout failures.