Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Invalid
-
0.15.0
-
None
-
None
Description
Using the example at https://github.com/apache/arrow/tree/master/rust/parquet is slow.
The snippet
let reader = SerializedFileReader::new(file).unwrap(); let mut iter = reader.get_row_iter(None).unwrap(); let start = Instant::now(); while let Some(record) = iter.next() {} let duration = start.elapsed(); println!("{:?}", duration);
Runs for 17sec for a ~160MB parquet file.
If there is a more effective way to load a parquet file, it would be nice to add it to the readme.
P.S.: My goal is to construct an ndarray from it, I'd be happy for any tips.