Major Description
ShuffleFetcherBlockIteratorSuite has tons of duplicate code, like:
val iterator = new ShuffleBlockFetcherIterator(
taskContext,
transfer,
blockManager,
blocksByAddress,
(_, in) => in,
48 * 1024 * 1024,
Int.MaxValue,
Int.MaxValue,
Int.MaxValue,
true,
false,
metrics,
false)
It's challenging to tell what the interesting parts are vs. what is just being set to some default/unused value.
Similarly but not as bad, there are 10 calls like:
verify(transfer, times(1)).fetchBlocks(any(), any(), any(), any(), any(), any())
and 7 like
when(transfer.fetchBlocks(any(), any(), any(), any(), any(), any())).thenAnswer ...
This can result in about 10% reduction in both lines and characters in the file:
# Before
> wc core/src/test/scala/org/apache/spark/storage/ShuffleBlockFetcherIteratorSuite.scala
1063 3950 43201 core/src/test/scala/org/apache/spark/storage/ShuffleBlockFetcherIteratorSuite.scala
# After
> wc core/src/test/scala/org/apache/spark/storage/ShuffleBlockFetcherIteratorSuite.scala
928 3609 39053 core/src/test/scala/org/apache/spark/storage/ShuffleBlockFetcherIteratorSuite.scala
It also helps readability:
val iterator = createShuffleBlockIteratorWithDefaults(
transfer,
blocksByAddress,
maxBytesInFlight = 1000L
)
Now I can clearly tell that maxBytesInFlight is the main parameter we're interested in here.
