Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-11687

[Rust][DataFusion] RepartitionExec Hanging

    XMLWordPrintableJSON

Details

    Description

      Found an interesting defect where the final partition of the `RepartitionExec::execute` thread spawner was consistently not being spawned via `tokio::spawn`. This meant that `RepartitionStream::poll_next` was sitting waiting forever for data that never arrived.

      It looks like a race condition where the `JoinHandle` was not being `await`ed and something strange going on with the internals of tokio like lazy evaluation?

      Attachments

        Activity

          People

            MikeSeddonAU Mike Seddon
            MikeSeddonAU Mike Seddon
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 7h 40m
                7h 40m