Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-13242

Moderately complex `when` expression causes code generation failure

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.6.0
    • 1.6.2, 2.0.0
    • SQL
    • None

    Description

      Moderately complex `when` expressions produce generated code that busts the 64KB method limit. This causes code generation to fail.

      Here's a test case exhibiting the problem: https://github.com/joehalliwell/spark/commit/4dbdf6e15d1116b8e1eb44822fd29ead9b7d817d

      I'm interested in working on a fix. I'm thinking it may be possible to split the expressions along the lines of SPARK-8443, but any pointers would be welcome!

      Attachments

        Activity

          People

            davies Davies Liu
            joehalliwell Joe Halliwell
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: