[DRILL-8059] Fragment starts late

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.13.0
• Component/s: Execution - RPC
• Assignee: Unassigned
• Reporter: wtf
• Created: 30/Nov/21 08:05
• Updated: 01/Dec/21 14:40

Description

One of the minor fragment starts late sometimes. All fragments transterred between the same node pair take muck longer time during that period.

Three profiles attached. The slow communication happens around 11:45:18~11:45:38 between node1 and node2.

On node1:

drillbit.log.1:2021-11-30 11:45:25,258 [1e5a61ab-1b3e-8956-ad52-6786ccece9e2:foreman] INFO o.a.d.e.work.foreman.FragmentsRunner - Sending remote fragments to node: node2, Fragments: [1e5a61ab-1b3e-8956-ad52-6786ccece9e2:1:17] 

On node2:

drillbit.log.3:2021-11-30 11:45:47,269 [BitServer-10] INFO o.a.d.e.w.b.ControlMessageHandler - 1e5a61ab-1b3e-8956-ad52-6786ccece9e2:1:17 remote fragment received

We can see that it takes more than 22 seconds to send the fragment.