[DRILL-1460] JsonReader fails reading files with decimal numbers and integers in the same field

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.6.0, 0.7.0
• Fix Version/s: 1.0.0
• Component/s: Storage - JSON
• Assignee: Jason Altekruse
• Reporter: Bhallamudi Venkata Siva Kamesh
• Created: 27/Sep/14 12:25
• Updated: 07/May/15 22:59
• Resolved: 07/May/15 18:04

Description

Used the following dataset : http://thecodebarbarian.wordpress.com/2014/03/28/plugging-usda-nutrition-data-into-mongodb

Executed the following query

select t.nutrients from dfs.usda.`usda.json` t limit 1;

and it failed with following exception

2014-09-27 17:48:39,421 [b9dfbb9b-29a9-425d-801c-2e418533525f:frag:0:0] ERROR o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error 0568d90a-d7df-4a5d-87e9-8b9f718dffa4: Screen received stop request sent.
java.lang.IllegalArgumentException: You tried to write a BigInt type when you are using a ValueWriter of type NullableFloat8WriterImpl.
	at org.apache.drill.exec.vector.complex.impl.AbstractFieldWriter.fail(AbstractFieldWriter.java:513) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.impl.AbstractFieldWriter.write(AbstractFieldWriter.java:145) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.impl.NullableFloat8WriterImpl.write(NullableFloat8WriterImpl.java:88) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:257) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:310) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:204) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.fn.JsonReader.write(JsonReader.java:134) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.fn.JsonReaderWithState.write(JsonReaderWithState.java:65) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.store.easy.json.JSONRecordReader2.next(JSONRecordReader2.java:111) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]

select t.nutrients[0].units from dfs.usda.`usda.json` t limit 1;

and it failed with following exception

2014-09-27 17:50:04,394 [9ee8a529-17fd-492f-9cba-2d1f5842eae1:frag:0:0] ERROR o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error c4c6bffd-b62b-4878-af1e-58db64453307: Screen received stop request sent.
java.lang.IllegalArgumentException: You tried to write a BigInt type when you are using a ValueWriter of type NullableFloat8WriterImpl.
	at org.apache.drill.exec.vector.complex.impl.AbstractFieldWriter.fail(AbstractFieldWriter.java:513) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.impl.AbstractFieldWriter.write(AbstractFieldWriter.java:145) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.impl.NullableFloat8WriterImpl.write(NullableFloat8WriterImpl.java:88) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:257) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:310) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:204) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.fn.JsonReader.write(JsonReader.java:134) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.vector.complex.fn.JsonReaderWithState.write(JsonReaderWithState.java:65) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.store.easy.json.JSONRecordReader2.next(JSONRecordReader2.java:111) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]
	at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:158) ~[drill-java-exec-0.6.0-incubating-SNAPSHOT-rebuffed.jar:0.6.0-incubating-SNAPSHOT]

Issue Links

• is duplicated by: DRILL-2716 Casting integer values from JSON file to float/double datatype fails.
• is related to: DRILL-2666 Query against json profile with large number of fragments fails