[ARROW-6154] [Rust] [Parquet] Too many open files (os error 24)

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Component/s: Rust
• External issue URL: https://github.com/apache/arrow/issues/22547
• Assignee: Unassigned
• Reporter: Yesh
• Created: 06/Aug/19 23:52
• Updated: 11/Jan/23 07:45
• Resolved: 26/Apr/21 11:23

Description

Used [rust]parquet-read binary to read a deeply nested parquet file and see the below stack trace. Unfortunately won't be able to upload file.

stack backtrace:

   0: std::panicking::default_hook::{{closure}}

   1: std::panicking::default_hook

   2: std::panicking::rust_panic_with_hook

   3: std::panicking::continue_panic_fmt

   4: rust_begin_unwind

   5: core::panicking::panic_fmt

   6: core::result::unwrap_failed

   7: parquet::util::io::FileSource<R>::new

   8: <parquet::file::reader::SerializedRowGroupReader<R> as parquet::file::reader::RowGroupReader>::get_column_page_reader

   9: <parquet::file::reader::SerializedRowGroupReader<R> as parquet::file::reader::RowGroupReader>::get_column_reader

  10: parquet::record::reader::TreeBuilder::reader_tree

  11: parquet::record::reader::TreeBuilder::reader_tree

  12: parquet::record::reader::TreeBuilder::reader_tree

  13: parquet::record::reader::TreeBuilder::reader_tree

  14: parquet::record::reader::TreeBuilder::reader_tree

  15: parquet::record::reader::TreeBuilder::build

  16: <parquet::record::reader::RowIter as core::iter::traits::iterator::Iterator>::next

  17: parquet_read::main

  18: std::rt::lang_start::{{closure}}

  19: std::panicking::try::do_call

  20: __rust_maybe_catch_panic

  21: std::rt::lang_start_internal

  22: main

Issue Links

• is fixed by: ARROW-9726 [Rust] [DataFusion] ParquetScanExec launches threads too early