Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-11855

Catalyst breaks backwards compatibility in branch-1.6

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Won't Fix
    • Affects Version/s: 1.6.0
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:
      None

      Description

      There's a number of APIs broken in catalyst 1.6.0. I'm trying to compile most cases:

      UnresolvedRelation's constructor has been changed from taking a Seq to a TableIdentifier. A deprecated constructor taking Seq would be needed to be backwards compatible.

       case class UnresolvedRelation(
      -    tableIdentifier: Seq[String],
      +    tableIdentifier: TableIdentifier,
           alias: Option[String] = None) extends LeafNode {
      

      It is similar with UnresolvedStar:

      -case class UnresolvedStar(table: Option[String]) extends Star with Unevaluable {
      +case class UnresolvedStar(target: Option[Seq[String]]) extends Star with Unevaluable {
      

      Catalog did get a lot of signatures changed too (because of TableIdentifier). Providing the older methods as deprecated also seems viable here.

      Spark 1.5 already broke backwards compatibility of part of catalyst API with respect to 1.4. I understand there are good reasons for some cases, but we should try to minimize backwards compatibility breakages for 1.x. Specially now that 2.x is on the horizon and there will be a near opportunity to remove deprecated stuff.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                smolav Santiago M. Mola
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: