Details
-
Improvement
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
None
-
None
-
None
Description
I have a very sparse protobuf message in my project, with thousands of fields.
In practise, most of the fields are all null values in one page.
But the repetition level and definition level takes lots of storage space.
Can parquet skip the storage of r level and d level for such all-null columns to save storage space?