[ARROW-13120] [Rust][Parquet] Cannot read multiple batches from parquet with string list column

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Later
• Component/s: Rust
• External issue URL: https://github.com/apache/arrow/issues/28822
• Assignee: Unassigned
• Reporter: Morgan Cassels
• Created: 18/Jun/21 17:05
• Updated: 11/Jan/23 08:31
• Resolved: 11/Jul/21 12:47

Description

This issue only occurs when the batch size < the number of rows in the table. The attached parquet `test.parquet` has 31430 rows and a single column containing string lists. This issue does not appear to occur for parquets with integer list columns.

  

#[test]
 fn failing_test() {
 let parquet_file_reader = get_test_reader("test.parquet");
 let mut arrow_reader = ParquetFileArrowReader::new(parquet_file_reader);
 let mut record_batches = Vec::new();
 let record_batch_reader = arrow_reader.get_record_reader(1024).unwrap();
 for batch in record_batch_reader {
   record_batches.push(batch);
 }
}

 

---- arrow::arrow_reader::tests::failing_test stdout ----
thread 'arrow::arrow_reader::tests::failing_test' panicked at 'Expected infallable creation of GenericListArray from ArrayDataRef failed: InvalidArgumentError("offsets do not start at zero")', arrow/src/array/array_list.rs:195:45
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace