Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7449

TotalNetworkThroughput in KrpcDataStreamSender is broken

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 3.0, Impala 2.12.0
    • Fix Version/s: Impala 3.1.0
    • Component/s: Distributed Exec
    • Labels:
      None
    • Epic Color:
      ghx-label-1

      Description

      The network throughput computation fails to take into account of the fact that multiple RPCs can be happening in parallel. Currently, the throughput is computed by (total bytes sent / total network time). The total network time is the aggregate of the network time observed of each RPC. This seems hard to understand (or wrong?) when there are drastically different throughput when sending to different hosts. It may be slightly easier to understand if we switch to measuring the observed network throughput of each individual RPC and use a summary counter or a histogram to record the throughput.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                kwho Michael Ho
                Reporter:
                kwho Michael Ho
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: