[PIG-3671] CONCAT operation bleeds into ToDate, making it ERROR

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 0.12.0
• Component/s: internal-udfs

Environment

Moonlight on the ocean last night was pretty.

• Assignee: Daniel Dai
• Reporter: Russell Jurney
• Created: 16/Jan/14 19:56
• Updated: 16/Jan/14 21:27
• Resolved: 16/Jan/14 21:27

Description

date_and_time = LOAD 'date_and_time.txt' AS (date:chararray, time:chararray);
date_time_concat = FOREACH date_and_time GENERATE CONCAT(CONCAT(date, ' '), time) AS date_time;
date_time_problem = FOREACH date_time_concat GENERATE ToDate(date_time) AS date_time:datetime;
dump date_time_problem

g.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing [POUserFunc (Name: POUserFunc(org.apache.pig.builtin.ToDateISO)[datetime] - scope-12 Operator Key: scope-12) children: null at []]: java.lang.IllegalArgumentException: Invalid format: "#date time"
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:338)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:378)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:298)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:282)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
Caused by: java.lang.IllegalArgumentException: Invalid format: "#date time"
at org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:683)
at org.apache.pig.builtin.ToDate.extractDateTime(ToDate.java:124)
at org.apache.pig.builtin.ToDateISO.exec(ToDateISO.java:38)
at org.apache.pig.builtin.ToDateISO.exec(ToDateISO.java:31)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:330)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNextDateTime(POUserFunc.java:422)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:329)