[DRILL-5860] projecting columns from Parquert - SYSTEM ERROR: ArrayIndexOutOfBoundsException

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.11.0
• Component/s: Storage - Parquet

Environment

Drill 1.11

• Assignee: Unassigned
• Reporter: N Campbell
• Created: 11/Oct/17 04:42
• Updated: 01/Feb/18 23:35

Description

Query attempts to project "select * from parquetfile" and encounters

Query which projects the columns by name does not.

Query which projects the columns by name and applies various cast expressions does.

Error: [MapR][DrillJDBCDriver](500165) Query execution error: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: ArrayIndexOutOfBoundsException

Fragment 0:0

[Error Id: 6c165788-d682-42f9-b5fc-d7c18177402b on csx00770.canlab.ibm.com:31010]

(org.apache.drill.common.exceptions.DrillRuntimeException) Error in parquet record reader.
 Message: 
 Hadoop path: /home/drill/data/gosales_parquet/HR/data/employee_history/000000_0
 Total records read: 0
 Row group index: 0
 Records in row group: 972
 Parquet Metadata: ParquetMetaData{FileMetaData{schema: message hive_schema

{ optional int32 employee_history_code; optional int32 employee_history_parent; optional int32 employee_code; optional int96 record_start_date; optional int96 record_end_date; optional int32 position_code; optional int96 position_start_date; optional int32 manager_code; optional binary manager (UTF8); optional binary manager_mb (UTF8); optional int32 branch_code; optional binary organization_code (UTF8); }

, metadata: {}}, blocks: [BlockMetaData{972, 42862 [ColumnMetaData

{UNCOMPRESSED [employee_history_code] INT32 [RLE, PLAIN, BIT_PACKED], 4}

, ColumnMetaData

{UNCOMPRESSED [employee_history_parent] INT32 [RLE, PLAIN_DICTIONARY, BIT_PACKED], 3935}

, ColumnMetaData

{UNCOMPRESSED [employee_code] INT32 [RLE, PLAIN, BIT_PACKED], 5787}

, ColumnMetaData

{UNCOMPRESSED [record_start_date] INT96 [RLE, PLAIN_DICTIONARY, BIT_PACKED], 9718}

, ColumnMetaData

{UNCOMPRESSED [record_end_date] INT96 [RLE, PLAIN_DICTIONARY, BIT_PACKED], 18576}

, ColumnMetaData

{UNCOMPRESSED [position_code] INT32 [RLE, PLAIN_DICTIONARY, BIT_PACKED], 19703}

, ColumnMetaData

{UNCOMPRESSED [position_start_date] INT96 [RLE, PLAIN_DICTIONARY, BIT_PACKED], 20673}

, ColumnMetaData

{UNCOMPRESSED [manager_code] INT32 [RLE, PLAIN_DICTIONARY, BIT_PACKED], 29675}

, ColumnMetaData

{UNCOMPRESSED [manager] BINARY [RLE, PLAIN_DICTIONARY, BIT_PACKED], 31515}

, ColumnMetaData

{UNCOMPRESSED [manager_mb] BINARY [RLE, PLAIN_DICTIONARY, BIT_PACKED], 36122}

, ColumnMetaData

{UNCOMPRESSED [branch_code] INT32 [RLE, PLAIN_DICTIONARY, BIT_PACKED], 40712}

, ColumnMetaData

{UNCOMPRESSED [organization_code] BINARY [RLE, PLAIN_DICTIONARY, BIT_PACKED], 41379}

]}]}
 org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.handleException():272
 org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next():299
 org.apache.drill.exec.physical.impl.ScanBatch.next():180
 org.apache.drill.exec.record.AbstractRecordBatch.next():119
 org.apache.drill.exec.record.AbstractRecordBatch.next():109
 org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
 org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():133
 org.apache.drill.exec.record.AbstractRecordBatch.next():162
 org.apache.drill.exec.record.AbstractRecordBatch.next():119
 org.apache.drill.exec.record.AbstractRecordBatch.next():109
 org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
 org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():133
 org.apache.drill.exec.record.AbstractRecordBatch.next():162
 org.apache.drill.exec.physical.impl.BaseRootExec.next():105
 org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():81
 org.apache.drill.exec.physical.impl.BaseRootExec.next():95
 org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():234
 org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():227
 java.security.AccessController.doPrivileged():-2
 javax.security.auth.Subject.doAs():422
 org.apache.hadoop.security.UserGroupInformation.doAs():1657
 org.apache.drill.exec.work.fragment.FragmentExecutor.run():(227
 org.apache.drill.common.SelfCleaningRunnable.run():38
 java.util.concurrent.ThreadPoolExecutor.runWorker():1142
 java.util.concurrent.ThreadPoolExecutor$Worker.run():617
 java.lang.Thread.run():748
 Caused By (java.lang.ArrayIndexOutOfBoundsException) null