[PIG-2374] streaming regression with dotNext

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.2
• Labels: hadoop023

Environment

hadoopApache Pig version 0.9.2.1111101150 (r1200499)
compiled Nov 10 2011, 19:50:15
-bash-3.1$ hadoop version
Hadoop 0.23.0.1111080202

Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23.0/hadoop-common-project/hadoop-common -r 1196973
Compiled by hadoopqa on Tue Nov 8 02:12:04 PST 2011
From source with checksum 4e42b2d96c899a98a8ab8c7cc23f27ae

• Hadoop Flags: Reviewed
• Assignee: Daniel Dai
• Reporter: Araceli Henley
• Created: 15/Nov/11 23:29
• Updated: 23/Jan/12 07:31
• Resolved: 07/Dec/11 20:25

Description

Streaming seems to be broken in dotNext. There are several tests that are failing.
The results from C below produce clean results.
The results from D which are streamed through CMD produce control characters on some of the output.

define CMD `perl GroupBy.pl '\t' 0` ship('/homes/monster/pigtest/pigtest_next/pigharness/dist/pig_harness/libexec/PigTest/GroupBy.pl');
A = load '/user/user1/pig/tests/data/singlefile/studenttab10k';
B = group A by $0;
C = foreach B generate flatten(A);
D = stream C through CMD;
store C into '/user/user1/pig/out/user1.1321117428/ComputeSpec_7_C.out';
store D into '/user/user1/pig/out/user1.1321117428/ComputeSpec_7_D.out';

Other streaming tests that fail with control characters:
EST FAILED <ComputeSpec_7>
TEST FAILED <ComputeSpec_8>
TEST FAILED <ComputeSpec_10>
TEST FAILED <ComputeSpec_11>
TEST FAILED <ComputeSpec_12>
TEST FAILED <JobManagement_2>
TEST FAILED <JobManagement_3>
TEST FAILED <StreamingIO_4>
TEST FAILED <NonStreaming_1>
TEST FAILED <MultiQuery_21>
...

Issue Links

• is broken by: HADOOP-6109 Handle large (several MB) text input lines in a reasonable amount of time