[SPARK-9232] Duplicate code in JSONRelation

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4.0
• Fix Version/s: 1.5.0
• Component/s: SQL
• Target Version/s: 1.5.0
• Assignee: Andrew Or
• Reporter: Andrew Or
• Created: 21/Jul/15 20:58
• Updated: 22/Jul/15 06:00
• Resolved: 22/Jul/15 06:00

Description

The following block appears identically in two places:

var success: Boolean = false
try {
  success = fs.delete(filesystemPath, true)
} catch {
  case e: IOException =>
    throw new IOException(
      s"Unable to clear output directory ${filesystemPath.toString} prior"
        + s" to writing to JSON table:\n${e.toString}")
}
if (!success) {
  throw new IOException(
    s"Unable to clear output directory ${filesystemPath.toString} prior"
      + s" to writing to JSON table.")
  }
}

https://github.com/apache/spark/blob/e5d2c37c68ac00a57c2542e62d1c5b4ca267c89e/sql/core/src/main/scala/org/apache/spark/sql/json/JSONRelation.scala#L72

https://github.com/apache/spark/blob/e5d2c37c68ac00a57c2542e62d1c5b4ca267c89e/sql/core/src/main/scala/org/apache/spark/sql/json/JSONRelation.scala#L131

Issue Links

• links to: [Github] Pull Request #7576 (andrewor14)