[PIG-3974] E2E test data generation fails in cluster mode

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.12.0
• Fix Version/s: 0.14.0
• Component/s: e2e harness
• Assignee: Lorand Bendig
• Reporter: Lorand Bendig
• Created: 30/May/14 21:06
• Updated: 21/Nov/14 05:58
• Resolved: 02/Jun/14 17:10

Description

When test-e2e-deploy target is called to generate data for e2e tests, Pig fails to execute fs -mkdir /user/pig/tests/data and returns with the following error:

Failed to generate data for testing: <Failed running /home/my/pig/trunk/test/e2e/pig/../../../bin/pig -e fs -mkdir /user/pig/tests/data>

This issue only happens if the parent directory /user/pig/tests doesn't exist. In this case org.apache.hadoop.fs.shell.Mkdir.processNonexistentPath() throws a PathNotFoundException in the background.

I suspect, that the full parent directories should be created along the path. LocalDeployer.pm uses mkdir -p, but ExistingClusterDeployer.pm doesn't which leads to this issue.