I found that when MROutputThread collecting the output of Reducer, it calls StreamKeyValUtil.splitKeyVal() and two local byte-arrays are allocated there for each line of output. Later these two byte-arrays are passed to variable key and val. There are twice memory copying here, one is the System.arraycopy() method, the other is inside key.set() / val.set().
This causes double times of memory copying for the whole output (may lead to higher CPU consumption), and frequent temporay object allocation.
|Status||Open [ 1 ]||Patch Available [ 10002 ]|
|Status||Patch Available [ 10002 ]||Resolved [ 5 ]|
|Fix Version/s||0.22.0 [ 12314184 ]|
|Resolution||Fixed [ 1 ]|
|Assignee||Ruibang He [ ruibang ]|
|Affects Version/s||0.22.0 [ 12314184 ]|