[SPARK-7436] Cannot implement nor use custom StandaloneRecoveryModeFactory implementations

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.1
• Fix Version/s: 1.3.2, 1.4.0
• Component/s: Deploy
• Target Version/s: 1.3.2, 1.4.0
• Assignee: Jacek Lewandowski
• Reporter: Jacek Lewandowski
• Created: 07/May/15 06:56
• Updated: 08/May/15 18:39
• Resolved: 08/May/15 18:39

Description

At least, this code fragment is buggy (Master.scala):

      case "CUSTOM" =>
        val clazz = Class.forName(conf.get("spark.deploy.recoveryMode.factory"))
        val factory = clazz.getConstructor(conf.getClass, Serialization.getClass)
          .newInstance(conf, SerializationExtension(context.system))
          .asInstanceOf[StandaloneRecoveryModeFactory]
        (factory.createPersistenceEngine(), factory.createLeaderElectionAgent(this))

Because here: val factory = clazz.getConstructor(conf.getClass, Serialization.getClass) it tries to find the constructor which accepts org.apache.spark.SparkConf and class of companion object of akka.serialization.Serialization and then it tries to instantiate newInstance(conf, SerializationExtension(context.system)) with instance of SparkConf and instance of Serialization class - not the companion objects.

Issue Links

• links to: [Github] Pull Request #5975 (jacek-lewandowski)
• links to: [Github] Pull Request #5976 (jacek-lewandowski)
• links to: [Github] Pull Request #5977 (jacek-lewandowski)