Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1717

Reduce backend TCP connections per-node to be independent of the number of backends

    XMLWordPrintableJSON

    Details

      Description

      We currently use up to |nodes|*|fragments| TCP connections per backend. Eventually, we should use a fixed number of connections over which all data stream sending can be multiplexed, but we will probably need to move away from Thrift before that happens, because we will need non-blocking IO to avoid deadlocks.

      In the meantime, we can use a fixed-size thread-pool per fragment, and at least only be linear in the number of fragments.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                henryr Henry Robinson
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: