Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
3.0.1
-
None
Description
Spark Version: 3.0.1
Description: The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefind' when the readBytes value is negative, as the attachment shows.
curl http:/hadoop001:18081/api/v1/applications/application_1601774913550_0225/stages/2/0/taskSummary
{
"quantiles" : [ 0.05, 0.25, 0.5, 0.75, 0.95 ],
"executorDeserializeTime" : [ 7.0, 357.0, 390.0, 484.0, 492.0 ],
...
"shuffleReadMetrics" :
{ "readBytes" : [ -2.0, 1775984.0, 1779759.0, 1781727.0, 1788426.0 ], "readRecords" : [ 2001.0, 2002.0, 2002.0, 2002.0, 2002.0 ], ... }