[DRILL-2666] Query against json profile with large number of fragments fails

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 0.8.0
• Fix Version/s: 0.9.0
• Component/s: Execution - Flow
• Assignee: Sudheesh Katkam
• Reporter: Krystal
• Created: 02/Apr/15 18:13
• Updated: 08/Apr/15 20:55
• Resolved: 08/Apr/15 20:55

Description

git.commit.id=417ad38f7239c6dc1ff0421cbd5a2358d1437c87

I have a json profile that contains about 300 fragments. When I try to run queries against it, the query would fail with "You tried to do a batch data read operation when you were in a state of STOP" error. Below is an example:

0: jdbc:drill:schema=dfs.hbase> select t3.majorId from (select t2.majorFragmentId majorId,t2.minorFragmentProfile.minorFragmentId minorId,flatten(t2.minorFragmentProfile.operatorProfile) oProfile from (select t1.col.majorFragmentId majorFragmentId,flatten(t1.col.minorFragmentProfile) minorFragmentProfile from (select flatten(t.fragmentprofile) col from `dfs.drillTestDir`.`/ui_profiles/cancelled_query2.json`t) t1) t2) t3 group by t3.majorId,t3.oProfile.operatorId,t3.oProfile.operatorType order by t3.majorId,t3.oProfile.operatorId;
Query failed: RemoteRpcException: Failure while running fragment., You tried to do a batch data read operation when you were in a state of STOP.  You can only do this type of operation when you are in a state of OK or OK_NEW_SCHEMA. [ 4de479a5-ce93-4a3f-89b4-3aa55481869c on qa-node114.qa.lab:31010 ]
[ 4de479a5-ce93-4a3f-89b4-3aa55481869c on qa-node114.qa.lab:31010 ]

For json profile files that contain smaller number of fragments, the queries run fine.

Issue Links

• relates to: DRILL-2476 Handle IterOutcome.STOP in buildSchema()
• relates to: DRILL-1460 JsonReader fails reading files with decimal numbers and integers in the same field