[DRILL-6197] Duplicate entries in inputProfiles of minor fragments for specific operators

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.12.0
• Fix Version/s: 1.13.0
• Component/s: Execution - Monitoring
• Labels: ready-to-commit
• Assignee: Kunal Khatua
• Reporter: Kunal Khatua
• Reviewer: Aman Sinha
• Created: 28/Feb/18 21:49
• Updated: 03/Mar/18 18:57
• Resolved: 03/Mar/18 18:57

Description

Minor fragments for the following operators show duplicate entries of the inputProfile (org.apache.drill.exec.ops.OperatorStats instance) when viewed in the Profile UI.
e.g

{
    ...
	"query": "select * from sys.version",
    ...
	        [ ...
			{
				"inputProfile": [{
					"records": 0,
					"batches": 0,
					"schemas": 0
				}],
				"operatorId": 0,
				"operatorType": 13,
				"setupNanos": 0,
				"processNanos": 0,
				"peakLocalMemoryAllocated": 27131904,
				"waitNanos": 0
			},
			{
				"inputProfile": [{
					"records": 1,
					"batches": 1,
					"schemas": 1
				}],
				"operatorId": 0,
				"operatorType": 13,
				"setupNanos": 0,
				"processNanos": 752448,
				"peakLocalMemoryAllocated": 27131904,
				"metric": [{
					"metricId": 0,
					"longValue": 178
				}],
				"waitNanos": 889492
			}]
	...
}

operatorType: 13 is the screen operator, for which there can be only one inputProfile.

It turns out that by default, all minor fragments' operators are provide a list of inputProfiles by org.apache.drill.exec.ops.FragmentStats.newOperatorStats(OpProfileDef, BufferAllocator). However, for the following 4 operators, the org.apache.drill.exec.physical.impl.BaseRootExec constructors also inject OperatorStats.

org.apache.drill.exec.proto.beans.CoreOperatorType.SCREEN
org.apache.drill.exec.proto.beans.CoreOperatorType.SINGLE_SENDER
org.apache.drill.exec.proto.beans.CoreOperatorType.BROADCAST_SENDER
org.apache.drill.exec.proto.beans.CoreOperatorType.HASH_PARTITION_SENDER

All updates to the inputProfiles are done by the latter, while the former only reports zero values.

The workaround is to have org.apache.drill.exec.ops.FragmentStats skip injecting the org.apache.drill.exec.ops.OperatorStats instance for these operators

Issue Links

• links to: GitHub Pull Request #1141