Details
-
Bug
-
Status: Closed
-
Blocker
-
Resolution: Fixed
-
None
-
None
-
None
Description
Fix the bugs for Spark quick start when turning on file group reader and positional merging flag.
List some issues found so far:
- [compatibility]When no positions are stored in the header, the read query failed. Idea behavior: use key based merging instead of failing.
- [compatibility]When a parquet file contains Avro records, the file group reader of spark job will check if the payload is the expected type; otherwise, it will throw.