Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3175

Cache the Kudu client between queries

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: Kudu_Impala
    • Fix Version/s: Impala 2.9.0
    • Component/s: Frontend
    • Labels:

      Description

      Right now each time we analyze a new kudu query and create a new plan we create a new client instance, meaning it will query the master for tablet locations again. If we cache the client between queries it will keep that metadata internally (and invalidate if/when needed).

        Issue Links

          Activity

          Hide
          caseyc casey added a comment -

          I talked to the Kudu team and they recommended waiting on this. Before doing any work they suggested collecting some profiling information to see how bad the problem was.

          If client caching is needed, the suggested using a single client for the frontend and maybe a pool of clients for the backend (though that was less clear).

          Show
          caseyc casey added a comment - I talked to the Kudu team and they recommended waiting on this. Before doing any work they suggested collecting some profiling information to see how bad the problem was. If client caching is needed, the suggested using a single client for the frontend and maybe a pool of clients for the backend (though that was less clear).
          Hide
          caseyc casey added a comment -

          Downgraded because the Kudu team expected that caching the FE client wouldn't help much. The expectation was "might shave off 1-2ms from our query which has an existing fixed cost of 70+ms".

          Show
          caseyc casey added a comment - Downgraded because the Kudu team expected that caching the FE client wouldn't help much. The expectation was "might shave off 1-2ms from our query which has an existing fixed cost of 70+ms".

            People

            • Assignee:
              Unassigned
              Reporter:
              dralves David Alves
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development