[SPARK-33431] Simplifying equality of the instance of Option and Some(value)

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 3.1.0
• Component/s: Spark Core, SQL
• Assignee: Unassigned
• Reporter: Yang Jie
• Created: 12/Nov/20 11:32
• Updated: 07/Dec/20 15:45
• Resolved: 07/Dec/20 15:45

Description

There are some code like "InstanceOfOption == Some(value)",

for example "TaskSetManager#handleSuccessfulTask" method:

def handleSuccessfulTask(tid: Long, result: DirectTaskResult[_]): Unit = {
  ...
  if (successful(index) && killedByOtherAttempt.contains(tid)) {
    ...
    val resultSizeAcc = result.accumUpdates.find(a =>
      a.name == Some(InternalAccumulator.RESULT_SIZE)) 
    ...
}

We need create a new Option instance "Some(InternalAccumulator.RESULT_SIZE)"  to compare equivalently with a.name.

In this scenario, can use the "Option#contains" method to avoid creating new option instance

 

Issue Links

• links to: [Github] Pull Request #30353 (LuciferYang)
• links to: [Github] Pull Request #30353 (LuciferYang)