Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-14220 Build and test Spark against Scala 2.12
  3. SPARK-22128

Update paranamer to 2.8 to avoid BytecodeReadingParanamer ArrayIndexOutOfBoundsException with Scala 2.12 + Java 8 lambda

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.0
    • Component/s: Spark Core
    • Labels:
      None
    • Target Version/s:

      Description

      Minor, but probably needs a JIRA.

      After updating to Scala 2.12 I encountered this issue pretty quickly in tests: https://github.com/paul-hammant/paranamer/issues/17

      java.lang.ArrayIndexOutOfBoundsException: ....
      at com.thoughtworks.paranamer.BytecodeReadingParanamer$ClassReader.accept(BytecodeReadingParanamer.java:563)
      ...
      

      Spark depends on jackson-module-paranamer 2.6.7 to match the other jackson deps, and that depends on paranamer 2.6. The bug above is fixed in 2.8.

      However I noticed that, really, Spark uses jackson-module-scala 2.6.7.1, and that in turn actually depends on jackson-module-paranamer 2.7.9 (kind of odd), which already depends on 2.8.

      So it seemed prudent to just not manually manage this dependency down. Then we still need to manage paranamer to 2.8 because Avro 1.7 pulls in 2.3.

      But it all seems to work in a quick test. And it's necessary to get 2.12 working.

        Attachments

          Activity

            People

            • Assignee:
              srowen Sean Owen
              Reporter:
              srowen Sean Owen
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: