[SPARK-14008] Cleanup/Extend the Vectorized Parquet Reader - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Done
Affects Version/s: None
Fix Version/s: None
Component/s: SQL
Labels:
None

Description

Per nongli's suggestions:

We should do these things:
1. Remove the non-vectorized parquet reader code.
2. Support the remaining types (just big decimals?)
3. Move the logic to determine if our parquet reader can be used to planning. Only complex types should fall back to the parquet-mr reader.

Attachments

Sub-Tasks

1.	Remove non-vectorized/unsafe-row parquet record reader	Resolved	Sameer Agarwal
2.	Investigate types that are not supported by vectorized parquet record reader	Resolved	Unassigned
3.	Extract VectorizedColumnReader from VectorizedParquetRecordReader	Resolved	Sameer Agarwal
4.	Support TimestampType in vectorized parquet reader	Resolved	Sameer Agarwal
5.	Support high-precision decimals in vectorized parquet reader	Resolved	Sameer Agarwal
6.	Explicitly identify/catch UnsupportedOperationException during parquet reader initialization	Resolved	Sameer Agarwal

Activity

People

Assignee:: Unassigned

Reporter:: Sameer Agarwal

Votes:: 0 Vote for this issue

Watchers:: 1 Start watching this issue

Dates

Created:: 18/Mar/16 18:19

Updated:: 12/Dec/22 18:11

Resolved:: 02/Nov/16 16:55