Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5238

Support transferring reservation between ReservationTrackers

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.9.0
    • Fix Version/s: Impala 2.9.0
    • Component/s: Backend
    • Labels:

      Description

      In order to support claiming reservation atomically (IMPALA-3748) and distributing it to different execution nodes, we need to add a transfer operation to ReservationTrackers.

      The idea is that we would claim all the reservation in a single shared "initial reservation" tracker, then individual nodes could claim their share from there. When a node is closed, and other node that executes after it depends on getting its reservation, it would have to deposit its initial reservation back into the "initial reservation" tracker (or transfer it directly to the other node).

      This operation should

        Issue Links

          Activity

          Hide
          tarmstrong Tim Armstrong added a comment -

          IMPALA-5238: transfer reservations between trackers

          This is a primitive needed to implement claiming and distribution
          of initial reservations. It supports transferring reservation between
          any two ReservationTrackers under the same query.

          Also remove the public DecreaseReservation() method, which is now
          mostly redundant and we have no plans to use.

          Testing:

          • Added a test that exercises transfer between trackers at different
            levels and with different relationships.

          Change-Id: I21f008abaf1aa4fcd2d854769a603b97589af3b3
          Reviewed-on: http://gerrit.cloudera.org:8080/6708
          Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
          Tested-by: Impala Public Jenkins

          M be/src/runtime/bufferpool/reservation-tracker-test.cc
          M be/src/runtime/bufferpool/reservation-tracker.cc
          M be/src/runtime/bufferpool/reservation-tracker.h
          3 files changed, 295 insertions, 74 deletions

          Show
          tarmstrong Tim Armstrong added a comment - IMPALA-5238 : transfer reservations between trackers This is a primitive needed to implement claiming and distribution of initial reservations. It supports transferring reservation between any two ReservationTrackers under the same query. Also remove the public DecreaseReservation() method, which is now mostly redundant and we have no plans to use. Testing: Added a test that exercises transfer between trackers at different levels and with different relationships. Change-Id: I21f008abaf1aa4fcd2d854769a603b97589af3b3 Reviewed-on: http://gerrit.cloudera.org:8080/6708 Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com> Tested-by: Impala Public Jenkins — M be/src/runtime/bufferpool/reservation-tracker-test.cc M be/src/runtime/bufferpool/reservation-tracker.cc M be/src/runtime/bufferpool/reservation-tracker.h 3 files changed, 295 insertions , 74 deletions

            People

            • Assignee:
              tarmstrong Tim Armstrong
              Reporter:
              tarmstrong Tim Armstrong
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development