[SPARK-22393] spark-shell can't find imported types in class constructors, extends clause

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.2, 2.1.2, 2.2.0
• Fix Version/s: 2.3.0
• Component/s: Spark Shell
• Assignee: Mark Petruska
• Reporter: Ryan Williams
• Created: 29/Oct/17 14:42
• Updated: 03/Apr/18 05:33
• Resolved: 01/Dec/17 11:14

Description

$ spark-shell
…
scala> import org.apache.spark.Partition
import org.apache.spark.Partition

scala> class P(p: Partition)
<console>:11: error: not found: type Partition
       class P(p: Partition)
                  ^

scala> class P(val index: Int) extends Partition
<console>:11: error: not found: type Partition
       class P(val index: Int) extends Partition
                                       ^

Any class that I import gives "not found: type ___" when used as a parameter to a class, or in an extends clause; this applies to classes I import from JARs I provide via --jars as well as core Spark classes as above.

This worked in 1.6.3 but has been broken since 2.0.0.

Issue Links

• links to: [Github] Pull Request #19846 (mpetruska)