Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2568

ExecuteStatement RPC (and beeswax query() RPC) should not block

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: Impala 2.0
    • Fix Version/s: None
    • Component/s: Clients
    • Labels:

      Description

      Even though the client api is intended to be non-blocking for statement execution requests, Impala still blocks for some part (or parts) of query execution. The client cannot know how long a request will block which makes setting a good timeout value very difficult.

      Clients now have these two options:

      1) Set an long enough timeout to allow for execution. The value could be 5 or more mins. But if a timeout occurs, the user needs to investigate if the query was still executing. Also a real network timeout means a very long wait interval before retrying the request.

      2) Don't set a timeout. Most of the time this works well but when a real networking problem happens the client will hang forever.

      https://github.com/cloudera/impyla/issues/7 shows an example of such problems. Impyla ended up going with option #2.

      I've found #2 to be a big burden. To avoid hanging, any request to the server needs to be done in a separate thread which is then monitored and timed out. Option #1 is an easier way of avoiding hangs.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                caseyc casey
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated: