[SPARK-27185] mapPartition to replace map to speedUp Dataset's toLocalIterator process

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Invalid
• Affects Version/s: 2.0.0, 2.2.0, 2.3.0, 2.4.0
• Component/s: SQL
• Labels: patch performance
• Flags: Patch, Important
• Assignee: Unassigned
• Reporter: angerszhu
• Created: 18/Mar/19 07:55
• Updated: 12/Dec/22 18:10
• Resolved: 21/Mar/19 08:02

Description

In my case, I will use DataSet's toLocalIterator function, and I found that underlying code can be improved,it can be changed from map to mapPartitionsInternal to speed Up the process of  decode data to Internal Row