[SPARK-33077] Spark 3 / Cats 2.2.0 classpath issue

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 3.0.1
• Component/s: Spark Core, Spark Shell
• Assignee: Unassigned
• Reporter: Sami Dalouche
• Created: 06/Oct/20 19:03
• Updated: 12/Dec/22 18:10
• Resolved: 07/Oct/20 08:07

Description

A small project with minimal dependencies as well as instructions on how to reproduce the issue is available at:

https://github.com/samidalouche/spark3-cats220

Executing this code works fine with cats 2.1.1 but fails with cats 2.2.0, which is quite surprising since the spark and cats dependencies are pretty much distinct from each other.

 

java.lang.NoSuchMethodError: 'void cats.kernel.CommutativeSemigroup.$init$(cats.kernel.CommutativeSemigroup)'
 at cats.UnorderedFoldable$$anon$1.<init>(UnorderedFoldable.scala:78)
 at cats.UnorderedFoldable$.<init>(UnorderedFoldable.scala:78)
 at cats.UnorderedFoldable$.<clinit>(UnorderedFoldable.scala)
 at cats.data.NonEmptyListInstances$$anon$2.<init>(NonEmptyList.scala:539)
 at cats.data.NonEmptyListInstances.<init>(NonEmptyList.scala:539)
 at cats.data.NonEmptyList$.<init>(NonEmptyList.scala:458)
 at cats.data.NonEmptyList$.<clinit>(NonEmptyList.scala)
 at catsspark.Boom$.assumeValid_$bang(boom.scala:19)
 at catsspark.Boom$.boom(boom.scala:14)
 ... 47 elided

Thanks in advance for looking into this.

I submitted the same issue to cat's bug tracker: https://github.com/typelevel/cats/issues/3628