Details
-
Task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
Impala 2.11.0
-
None
-
ghx-label-9
Description
It was observed while inserting large amounts of data into a Kudu table Exchange operator was running slow, query profile showed a big portion of the time was spent in memory allocation in the buffer pool
EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 48s853ms, % non-child: 0.23%) - ConvertRowBatchTime: 20s289ms - PeakMemoryUsage: 19.53 MB (20483562) - RowsReturned: 575.30M (575298780) - RowsReturnedRate: 27.10 K/sec Buffer pool: - AllocTime: 2h53m - CumulativeAllocationBytes: 261.26 GB (280526643200) - CumulativeAllocations: 13.70M (13697590) - PeakReservation: 18.06 MB (18939904) - PeakUnpinnedBytes: 0 - PeakUsedReservation: 18.06 MB (18939904) - ReadIoBytes: 0 - ReadIoOps: 0 (0) - ReadIoWaitTime: 0.000ns - WriteIoBytes: 0 - WriteIoOps: 0 (0) - WriteIoWaitTime: 0.000ns RecvrSide: BytesReceived(8m32s): 20.91 GB, 37.03 GB, 45.62 GB, 53.22 GB, 60.17 GB, 66.30 GB, 71.60 GB, 76.59 GB, 81.36 GB, 86.03 GB, 90.35 GB, 94.30 GB, 98.17 GB, 101.98 GB, 105.58 GB, 109.08 GB, 112.33 GB, 115.47 GB, 118.45 GB, 121.30 GB, 124.09 GB, 126.74 GB, 129.26 GB, 131.88 GB, 134.41 GB, 136.85 GB, 139.32 GB, 141.77 GB, 144.23 GB, 146.71 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB - FirstBatchArrivalWaitTime: 1s071ms - TotalBytesReceived: 148.30 GB (159234237617) - TotalGetBatchTime: 5h53m - DataArrivalTimer: 5h52m SenderSide: - DeserializeRowBatchTime: 3h4m - NumBatchesArrived: 6.85M (6848795) - NumBatchesDeferred: 99.67K (99667) - NumBatchesEnqueued: 6.85M (6848795) - NumBatchesReceived: 6.85M (6848795) - NumEarlySenders: 0 (0) - NumEosReceived: 0 (0)