Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-9726

[Rust] [DataFusion] ParquetScanExec launches threads too early

    XMLWordPrintableJSON

Details

    Description

      ParquetScanExec launches threads in partitions() ahead of execute() being called on those partitions. This results on "too many open files" when there are many partitions.

      Attachments

        Issue Links

          Activity

            People

              andygrove Andy Grove
              andygrove Andy Grove
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m