Description
we call getInt on spark.executor.memory but this is a formatted string, example here: http://spark.apache.org/docs/1.0.1/configuration.html as such, I get:
2014-12-08 03:04:48,114 WARN [HiveServer2-Handler-Pool: Thread-34]: spark.SetSparkReducerParallelism (SetSparkReducerParallelism.java:process(141)) - Failed to create spark client. java.lang.NumberFormatException: For input string: "23000m" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:492) at java.lang.Integer.parseInt(Integer.java:527) at scala.collection.immutable.StringLike$class.toInt(StringLike.scala:229) at scala.collection.immutable.StringOps.toInt(StringOps.scala:31) at org.apache.spark.SparkConf$$anonfun$getInt$2.apply(SparkConf.scala:184) at org.apache.spark.SparkConf$$anonfun$getInt$2.apply(SparkConf.scala:184) at scala.Option.map(Option.scala:145) at org.apache.spark.SparkConf.getInt(SparkConf.scala:184) at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.getMemoryAndCores(SparkSessionImpl.java:72)