[ARROW-6189] [Rust] [Parquet] Plain encoded boolean column chunks limited to 2048 values

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 0.14.1
• Fix Version/s: 5.0.0
• Component/s: Rust
• External issue URL: https://github.com/apache/arrow/issues/22581
• Assignee: Unassigned
• Reporter: Simon Jones
• Created: 09/Aug/19 16:58
• Updated: 11/Jan/23 07:45
• Resolved: 26/Apr/21 11:24

Description

encoding::PlainEncoder::new creates a BitWriter with 256 bytes of storage, which limits the data page size that can be used.

I suggest that in

impl Encoder<BoolType> for PlainEncoder<BoolType>

the return value of put_value is tested and the BitWriter flushed+cleared whenever it runs out of space.