[DRILL-2566] Query nested arrays from JSON data results in Exception

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.8.0
• Fix Version/s: Future
• Component/s: Storage - JSON
• Assignee: Unassigned
• Reporter: Khurram Faraaz
• Created: 25/Mar/15 21:08
• Updated: 01/Oct/15 16:38

Description

Query over nested data (nested arrays) in JSON data file results in an Exception.

0: jdbc:drill:> use dfs.tmp;
+------------+------------+
|     ok     |  summary   |
+------------+------------+
| true       | Default schema changed to 'dfs.tmp' |
+------------+------------+
1 row selected (0.111 seconds)
0: jdbc:drill:> select * from `nestdJsnArry.json`;
Query failed: Query stopped., Needed to be in state INIT or IN_FLOAT8 but in mode IN_BIGINT [ 6b444a96-a331-49a4-b221-f1b2f96f4a81 on centos-01.qa.lab:31010 ]

Error: exception while executing query: Failure while executing query. (state=,code=0)

version details are

0: jdbc:drill:> select * from sys.version;
+------------+----------------+-------------+-------------+------------+
| commit_id  | commit_message | commit_time | build_email | build_time |
+------------+----------------+-------------+-------------+------------+
| f658a3c513ddf7f2d1b0ad7aa1f3f65049a594fe | DRILL-2209 Insert ProjectOperator with MuxExchange | 09.03.2015 @ 01:49:18 EDT | Unknown     | 09.03.2015 @ 04:52:49 EDT |
+------------+----------------+-------------+-------------+------------+

JSON data that I used in my test was

{"outkey":[[1,2,3,-1,0,9999,99999999,10.738],["a","p","d","e","w","o"],["string1","string2","string3","longer string","this is a test string"],[[-1,0,10,9999999,99999,879,-5678],["a",null]]]}

Stack trace snippet from drillbit.log

java.lang.IllegalStateException: Needed to be in state INIT or IN_FLOAT8 but in mode IN_BIGINT
        at org.apache.drill.exec.vector.complex.impl.RepeatedListWriter.float8(RepeatedListWriter.java:428) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:418) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:393) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:256) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.vector.complex.fn.JsonReader.writeDataSwitch(JsonReader.java:208) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.vector.complex.fn.JsonReader.writeToVector(JsonReader.java:182) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.vector.complex.fn.JsonReader.write(JsonReader.java:156) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.store.easy.json.JSONRecordReader.next(JSONRecordReader.java:125) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:165) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:97) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:57) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:121) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:303) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
2015-03-25 20:57:13,142 [2aecded5-f5d1-ef1c-ade6-870481f29fdb:frag:0:0] INFO  o.a.drill.exec.work.foreman.Foreman - State change requested.  RUNNING --> FAILED
org.apache.drill.exec.rpc.RemoteRpcException: Failure while running fragment., Needed to be in state INIT or IN_FLOAT8 but in mode IN_BIGINT [ a50def32-82b0-4811-9a5c-809fad89f806 on centos-01.qa.lab:31010 ]
[ a50def32-82b0-4811-9a5c-809fad89f806 on centos-01.qa.lab:31010 ]

        at org.apache.drill.exec.work.foreman.QueryManager.statusUpdate(QueryManager.java:95) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.QueryManager$RootStatusReporter.statusChange(QueryManager.java:154) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:114) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:110) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.internalFail(FragmentExecutor.java:176) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:123) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:303) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
2015-03-25 20:57:13,150 [2aecded5-f5d1-ef1c-ade6-870481f29fdb:frag:0:0] INFO  o.a.drill.exec.work.foreman.Foreman - State change requested.  FAILED --> COMPLETED
2015-03-25 20:57:13,150 [2aecded5-f5d1-ef1c-ade6-870481f29fdb:frag:0:0] WARN  o.a.drill.exec.work.foreman.Foreman - Dropping request to move to COMPLETED state as query is already at FAILED state (which is terminal).
2015-03-25 20:57:13,151 [2aecded5-f5d1-ef1c-ade6-870481f29fdb:frag:0:0] ERROR o.a.drill.exec.work.foreman.Foreman - Error 4243bad2-4a57-49a3-bca1-1db68f835bdf: RemoteRpcException: Failure while running fragment., Needed to be in state INIT or IN_FLOAT8 but in mode IN_BIGINT [ a50def32-82b0-4811-9a5c-809fad89f806 on centos-01.qa.lab:31010 ]
[ a50def32-82b0-4811-9a5c-809fad89f806 on centos-01.qa.lab:31010 ]

org.apache.drill.exec.rpc.RemoteRpcException: Failure while running fragment., Needed to be in state INIT or IN_FLOAT8 but in mode IN_BIGINT [ a50def32-82b0-4811-9a5c-809fad89f806 on centos-01.qa.lab:31010 ]
[ a50def32-82b0-4811-9a5c-809fad89f806 on centos-01.qa.lab:31010 ]

        at org.apache.drill.exec.work.foreman.QueryManager.statusUpdate(QueryManager.java:95) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.QueryManager$RootStatusReporter.statusChange(QueryManager.java:154) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:114) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:110) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.internalFail(FragmentExecutor.java:176) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:123) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:303) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]

Issue Links

• is related to: DRILL-2350 Star query failed with exception on JSON data