Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8947 Improve expression type coercion, casting & checking
  3. SPARK-9306

SortMergeJoin crashes when trying to join on unsortable columns

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.5.0
    • 1.5.0
    • SQL
    • None

    Description

      When sort merge join is enabled, it's possible to crash at runtime when trying to join on two StructFields: the problem is that we don't support ordering by struct fields or arrays. When the join columns are unsortable then we should not choose the SMJ join strategy.

      Attachments

        Activity

          People

            viirya L. C. Hsieh
            joshrosen Josh Rosen
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: