[PIG-4397] CSVExcelStorage incorrect output if last field value is null

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.15.0

Environment

Running the Pig version bundled with HDP 2.1.2: 0.12.1.2.1.2.0-402

• Hadoop Flags: Reviewed
• Assignee: Daniel Dai
• Reporter: Niels Basjes
• Created: 26/Jan/15 21:57
• Updated: 07/Jun/15 03:48
• Resolved: 25/Feb/15 01:45

Description

I have the following input:

one two
three
 four

I run this code

Lines =
    LOAD 'test.log' USING PigStorage(' ') 
    AS ( First:chararray , Second:chararray );

DUMP Lines;

STORE Lines INTO 'Lines'
USING org.apache.pig.piggybank.storage.CSVExcelStorage('\t', 'NO_MULTILINE', 'WINDOWS', 'WRITE_OUTPUT_HEADER');

The output from the DUMP is correct:

(one,two)
(three,)
(,four)

The output from the CSVExcelStorage is incorrect:

First   Second
one     two
three   three
        four

The problem is that if the last field is a null then the previous value is repeated incorrectly (in this case 'three').