Major Description
Division of BigDecimals doesn't work because we're not passing scale and rounding information in divide() method. In cases like 10/3 we'll get ArithmeticException:
Pig script:
grunt> A = LOAD 'decimaltest/f1' USING PigStorage(',') AS (id,col1:bigdecimal,col2:bigdecimal);
grunt> B = foreach A generate col1, col2, col1/col2;
grunt> dump B
Input file content:
1,10.0,3
2,51651351.13153143512,10.00
3,252525.252525,123.456
Output with bigdecimal type in the schema:
java.lang.Exception: org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing [Divide (Name: Divide[bigdecimal] - scope-34 Operator Key: scope-34) children: [[POProject (Name: Project[bigdecimal][0] - scope-32 Operator Key: scope-32) children: null at []], [POProject (Name: Project[bigdecimal][1] - scope-33 Operator Key: scope-33) children: null at []]] at []]: java.lang.ArithmeticException: Non-terminating decimal expansion; no exact representable decimal result.
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing [Divide (Name: Divide[bigdecimal] - scope-34 Operator Key: scope-34) children: [[POProject (Name: Project[bigdecimal][0] - scope-32 Operator Key: scope-32) children: null at []], [POProject (Name: Project[bigdecimal][1] - scope-33 Operator Key: scope-33) children: null at []]] at []]: java.lang.ArithmeticException: Non-terminating decimal expansion; no exact representable decimal result.
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:364)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:404)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:321)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:280)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:275)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:65)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ArithmeticException: Non-terminating decimal expansion; no exact representable decimal result.
at java.math.BigDecimal.divide(BigDecimal.java:1616)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Divide.divide(Divide.java:75)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Divide.genericGetNext(Divide.java:133)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Divide.getNextBigDecimal(Divide.java:166)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:353)
... 14 more
Output with double in the schema:
(10.0,3.0,3.3333333333333335)
(5.165135113153143E7,10.0,5165135.113153143)
(252525.252525,123.456,2045.467636445373)