[SPARK-45737] Remove unnecessary `.toArray[InternalRow]` in `SparkPlan#executeTake` function.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Yang Jie
• Reporter: Yang Jie
• Created: 31/Oct/23 08:49
• Updated: 31/Oct/23 15:56
• Resolved: 31/Oct/23 15:56

Description

if (takeFromEnd) {
  while (buf.length < n && i < res.length) {
    val rows = decodeUnsafeRows(res(i)._2)
    if (n - buf.length >= res(i)._1) {
      buf.prependAll(rows.toArray[InternalRow])
    } else {
      val dropUntil = res(i)._1 - (n - buf.length)
      // Same as Iterator.drop but this only takes a long.
      var j: Long = 0L
      while (j < dropUntil) { rows.next(); j += 1L}
      buf.prependAll(rows.toArray[InternalRow])
    }
    i += 1
  }
} else {
  while (buf.length < n && i < res.length) {
    val rows = decodeUnsafeRows(res(i)._2)
    if (n - buf.length >= res(i)._1) {
      buf ++= rows.toArray[InternalRow]
    } else {
      buf ++= rows.take(n - buf.length).toArray[InternalRow]
    }
    i += 1
  }
} 

In the above code, the input parameters of `mutable.Buffer#prependAll` and `mutable.Growable#++=` functions are `IterableOnce`, and the type of rows is `Iterator[InternalRow]`, which inherits from `IterableOnce`, so there is no need to cast to an array of InternalRow anymore.
 
 
 
 
 

Issue Links

• links to: GitHub Pull Request #43599