Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-16498

[C++] Fix potential deadlock in arrow::compute::TaskScheduler

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 9.0.0
    • C++

    Description

      An extremely simplified version of the task scheduler's ScheduleMore method it looks something like:

      void ScheduleMore(int num_to_schedule) {
        tasks_that_need_running_.fetch_add(num_to_schedule);
        if (!weak_lock.lock()) {
          // If someone else is scheduling then return early
          return;
        }
        auto tasks = PickTasks();
        weak_lock.unlock();
      }
      

      It is possible for one thread to have the lock, and find 0 tasks. But then, before it gives up the lock, another thread adds tasks and fails to acquire the lock. Neither thread will schedule anything even though there are tasks to run. This can lead to deadlock.

      The proposed PR changes the logic to (still extremely simplified):

      void ScheduleMore(int num_to_schedule) {
        tasks_that_need_running_.fetch_add(num_to_schedule);
        tasks_added_recently.store(true);
        if (!weak_lock.lock()) {
          // If someone else is scheduling then return early
          return;
        }
        auto tasks = PickTasks();
        if (tasks_added_recently.compare_exchange_strong(true, false)) {
          if (tasks.empty()) {
            ScheduleMore();
          }
        }
        weak_lock.unlock();
      }
      

      Attachments

        Issue Links

          Activity

            People

              westonpace Weston Pace
              westonpace Weston Pace
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h
                  1h