[SPARK-26383] NPE when use DataFrameReader.jdbc with wrong URL

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 3.0.0
• Component/s: Spark Core
• Assignee: clouds
• Reporter: clouds
• Created: 17/Dec/18 11:00
• Updated: 12/Dec/22 18:10
• Resolved: 07/Jan/19 14:59

Description

When passing wrong url to jdbc:

val opts = Map(
  "url" -> "jdbc:mysql://localhost/db",
  "dbtable" -> "table",
  "driver" -> "org.postgresql.Driver"
)
var df = spark.read.format("jdbc").options(opts).load

It would throw an NPE instead of complaining about connection failed. (Note url and driver not matched here)

Exception in thread "main" java.lang.NullPointerException
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:71)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
    at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
    at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
    at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)

as postgresql jdbc driver document saying, The driver should return "null" if it realizes it is the wrong kind of driver to connect to the given URL.

while ConnectionFactory would not check if conn is null.

val conn: Connection = JdbcUtils.createConnectionFactory(options)()

 and trying to close the conn anyway

try {
  ...
} finally {
  conn.close()
}

Issue Links

• links to: GitHub Pull Request #23464