Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-14220 Build and test Spark against Scala 2.12
  3. SPARK-22128

Update paranamer to 2.8 to avoid BytecodeReadingParanamer ArrayIndexOutOfBoundsException with Scala 2.12 + Java 8 lambda

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.0
    • Component/s: Spark Core
    • Labels:
      None
    • Target Version/s:

      Description

      Minor, but probably needs a JIRA.

      After updating to Scala 2.12 I encountered this issue pretty quickly in tests: https://github.com/paul-hammant/paranamer/issues/17

      java.lang.ArrayIndexOutOfBoundsException: ....
      at com.thoughtworks.paranamer.BytecodeReadingParanamer$ClassReader.accept(BytecodeReadingParanamer.java:563)
      ...
      

      Spark depends on jackson-module-paranamer 2.6.7 to match the other jackson deps, and that depends on paranamer 2.6. The bug above is fixed in 2.8.

      However I noticed that, really, Spark uses jackson-module-scala 2.6.7.1, and that in turn actually depends on jackson-module-paranamer 2.7.9 (kind of odd), which already depends on 2.8.

      So it seemed prudent to just not manually manage this dependency down. Then we still need to manage paranamer to 2.8 because Avro 1.7 pulls in 2.3.

      But it all seems to work in a quick test. And it's necessary to get 2.12 working.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                srowen Sean R. Owen
                Reporter:
                srowen Sean R. Owen
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: