[DRILL-1660] JSON : selecting from an array of empty, null maps results in an exception

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.7.0
• Component/s: Storage - JSON
• Assignee: Parth Chandra
• Reporter: Rahul Kumar Challapalli
• Created: 07/Nov/14 18:34
• Updated: 06/May/15 01:34
• Resolved: 30/Nov/14 19:44

Description

git.commit.id.abbrev=60aa446
I ran the below test against the branch from Jason's github fork which has some patches for bugs related to flatten which are not yet merged into the master.

Dataset 1 :

{
  "id":1,
  "evnts":[
    {},
    {"key":null}
  ]
}

Queries :

0: jdbc:drill:schema=dfs.drillTestDir> select id, evnts from `json_kvgenflatten/empty-null-map.json`;
+------------+------------+
|     id     |   evnts    |
+------------+------------+
java.lang.IndexOutOfBoundsException: DrillBuf(ridx: 0, widx: 0, cap: 0/0, unwrapped: DrillBuf(ridx: 0, widx: 17, cap: 17/17, unwrapped: DrillBuf(ridx: 103, widx: 103, cap: 103/103, unwrapped: UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap: 103/103))))).slice(0, 8)
	at io.netty.buffer.DrillBuf.<init>(DrillBuf.java:94)
	at io.netty.buffer.DrillBuf.slice(DrillBuf.java:293)
	at org.apache.drill.exec.vector.UInt4Vector.load(UInt4Vector.java:179)
	at org.apache.drill.exec.vector.complex.RepeatedMapVector.load(RepeatedMapVector.java:417)
	at org.apache.drill.exec.record.RecordBatchLoader.load(RecordBatchLoader.java:91)
	at org.apache.drill.jdbc.DrillCursor.next(DrillCursor.java:102)
	at net.hydromatic.avatica.AvaticaResultSet.next(AvaticaResultSet.java:187)
	at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2503)
	at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
	at sqlline.SqlLine.print(SqlLine.java:1809)
	at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
	at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
	at sqlline.SqlLine.dispatch(SqlLine.java:889)
	at sqlline.SqlLine.begin(SqlLine.java:763)
	at sqlline.SqlLine.start(SqlLine.java:498)
	at sqlline.SqlLine.main(SqlLine.java:460)
0: jdbc:drill:schema=dfs.drillTestDir> select id, flatten(evnts) from `json_kvgenflatten/empty-null-map.json`;
+------------+------------+
|     id     |   EXPR$1   |
+------------+------------+
| 1          | {}         |
| 1          | {}         |
+------------+------------+
2 rows selected (0.193 seconds)

Dataset 2:

{"id":1,"evnts":[]}

Queries :

0: jdbc:drill:schema=dfs.drillTestDir> select id, evnts from `json_kvgenflatten/temp.json`;
+------------+------------+
|     id     |   evnts    |
+------------+------------+
| 1          | null       |
+------------+------------+
1 row selected (0.12 seconds)
0: jdbc:drill:schema=dfs.drillTestDir> select id, flatten(evnts) from `json_kvgenflatten/temp.json`;
Query failed: Failure while running fragment., org.apache.drill.exec.vector.NullableIntVector cannot be cast to org.apache.drill.exec.vector.RepeatedVector [ cb7dfdf1-1f48-4fa1-ac73-6ffbbba0e32a on qa-node191.qa.lab:31010 ]


Error: exception while executing query: Failure while executing query. (state=,code=0)

Dataset 3:

{"id":1,"evnts":[{}]}

Queries :

0: jdbc:drill:schema=dfs.drillTestDir>      select id, evnts from `json_kvgenflatten/temp1.json`;
+------------+------------+
|     id     |   evnts    |
+------------+------------+
java.lang.IndexOutOfBoundsException: DrillBuf(ridx: 0, widx: 0, cap: 0/0, unwrapped: DrillBuf(ridx: 0, widx: 17, cap: 17/17, unwrapped: DrillBuf(ridx: 103, widx: 103, cap: 103/103, unwrapped: UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap: 103/103))))).slice(0, 8)
	at io.netty.buffer.DrillBuf.<init>(DrillBuf.java:94)
	at io.netty.buffer.DrillBuf.slice(DrillBuf.java:293)
	at org.apache.drill.exec.vector.UInt4Vector.load(UInt4Vector.java:179)
	at org.apache.drill.exec.vector.complex.RepeatedMapVector.load(RepeatedMapVector.java:417)
	at org.apache.drill.exec.record.RecordBatchLoader.load(RecordBatchLoader.java:91)
	at org.apache.drill.jdbc.DrillCursor.next(DrillCursor.java:102)
	at net.hydromatic.avatica.AvaticaResultSet.next(AvaticaResultSet.java:187)
	at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2503)
	at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
	at sqlline.SqlLine.print(SqlLine.java:1809)
	at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
	at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
	at sqlline.SqlLine.dispatch(SqlLine.java:889)
	at sqlline.SqlLine.begin(SqlLine.java:763)
	at sqlline.SqlLine.start(SqlLine.java:498)
	at sqlline.SqlLine.main(SqlLine.java:460)
0: jdbc:drill:schema=dfs.drillTestDir>      select id, flatten(evnts) from `json_kvgenflatten/temp1.json`;
+------------+------------+
|     id     |   EXPR$1   |
+------------+------------+
| 1          | {}         |
+------------+------------+
1 row selected (0.179 seconds)

I attached the error log for the first dataset. Let me know if you need anything else

Issue Links

• relates to: DRILL-1539 RepeatedMapVector maintains incorrect last non-empty element index.