Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
0.9.9
Description
While testing the assembly cost, I found an issue with certain types of columns (namely array items with union types). Upon querying, an exception was thrown:
Caused by: java.lang.IllegalAccessError: no more values, column index: 924 at org.apache.asterix.column.assembler.RepeatedPrimitiveValueAssembler.next(RepeatedPrimitiveValueAssembler.java:44) ~[classes/:?] at org.apache.asterix.column.operation.query.ColumnAssembler.nextValue(ColumnAssembler.java:80) ~[classes/:?] at org.apache.asterix.column.tuple.QueryColumnTupleReference.getAssembledValue(QueryColumnTupleReference.java:133) ~[classes/:?] at org.apache.asterix.column.operation.query.QueryColumnTupleProjector.getAssembledValue(QueryColumnTupleProjector.java:107) ~[classes/:?] at org.apache.asterix.column.operation.query.QueryColumnTupleProjector.project(QueryColumnTupleProjector.java:84) ~[classes/:?] at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.writeTupleToOutput(IndexSearchOperatorNodePushable.java:390) ~[classes/:?] at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.writeSearchResults(IndexSearchOperatorNodePushable.java:270) ~[classes/:?] at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.searchAllPartitions(IndexSearchOperatorNodePushable.java:459) ~[classes/:?] at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.nextFrame(IndexSearchOperatorNodePushable.java:312) ~[classes/:?] at org.apache.hyracks.api.dataflow.EnforceFrameWriter.nextFrame(EnforceFrameWriter.java:76) ~[classes/:?] at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:94) ~[classes/:?] at org.apache.hyracks.algebricks.runtime.operators.std.EmptyTupleSourceRuntimeFactory$1.open(EmptyTupleSourceRuntimeFactory.java:55) ~[classes/:?] at org.apache.hyracks.api.dataflow.EnforceFrameWriter.open(EnforceFrameWriter.java:59) ~[classes/:?] at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$SourcePushRuntime.initialize(AlgebricksMetaOperatorDescriptor.java:105) ~[classes/:?] at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$0(SuperActivityOperatorNodePushable.java:236) ~[classes/:?] at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java) ~[?:?]
Schema of the affected column(s)
root |-- geo: object <level: 1> | |-- coordinates: array <level: 2> | | |-- item: union <level: 3> | | | |-- bigint: bigint <level: 3, index: 924> | | | |-- double: double <level: 3, index: 888>
After Investigation:
The problem appears in a certain circumstance where the first N pages are written with the a schema as follows:
root
|-- geo: object <level: 1>
| |-- coordinates: array <level: 2>
| | |-- item: double <level: 3, index: 888>
Then, for all subsequent pages (i.e., N+M), the schema changes to:
root |-- geo: object <level: 1> | |-- coordinates: array <level: 2> | | |-- item: union <level: 3> | | | |-- bigint: bigint <level: 3, index: 924> | | | |-- double: double <level: 3, index: 888>
So the first N pages are missing column 924. However, the component's (with # of pages > N) schema contains the information of column 924. The assembler was blindly setting the delegate for the array coordinates (from the schema) to column 924 – a newer column for the array coordinates. Hence, column 924 does not contain all information necessary to assemble coordinates.
This is not a union issue per-se, but could also appear with objects with new fields with array types.