Details
-
Bug
-
Status: Closed
-
Critical
-
Resolution: Won't Fix
-
1.6.0
-
None
-
None
Description
There's a number of APIs broken in catalyst 1.6.0. I'm trying to compile most cases:
UnresolvedRelation's constructor has been changed from taking a Seq to a TableIdentifier. A deprecated constructor taking Seq would be needed to be backwards compatible.
case class UnresolvedRelation( - tableIdentifier: Seq[String], + tableIdentifier: TableIdentifier, alias: Option[String] = None) extends LeafNode {
It is similar with UnresolvedStar:
-case class UnresolvedStar(table: Option[String]) extends Star with Unevaluable { +case class UnresolvedStar(target: Option[Seq[String]]) extends Star with Unevaluable {
Catalog did get a lot of signatures changed too (because of TableIdentifier). Providing the older methods as deprecated also seems viable here.
Spark 1.5 already broke backwards compatibility of part of catalyst API with respect to 1.4. I understand there are good reasons for some cases, but we should try to minimize backwards compatibility breakages for 1.x. Specially now that 2.x is on the horizon and there will be a near opportunity to remove deprecated stuff.
Attachments
Issue Links
- relates to
-
SPARK-11780 Provide type aliases in org.apache.spark.sql.types for backwards compatibility
- Resolved
- links to