[SPARK-45391] spark-connect-repl is not working on macOS

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.5.0
• Component/s: Connect Contrib

Environment

MacBook M2

cs version
2.1.7

scala -version
Scala code runner version 2.12.18 – Copyright 2002-2023, LAMP/EPFL and Lightbend, Inc.

• Assignee: Unassigned
• Reporter: Vu Tan
• Created: 01/Oct/23 08:39
• Updated: 04/Oct/23 17:05

Description

I followed https://spark.apache.org/docs/latest/spark-connect-overview.html#use-spark-connect-for-interactive-analysis to try spark-connect-repl on my local PC but got the following error:

 

---------------------------

spark-connect-repl
Exception in thread "main" java.lang.NoClassDefFoundError: org/sparkproject/connect/client/com/google/common/io/BaseEncoding
    at org.sparkproject.connect.client.io.grpc.Metadata.<clinit>(Metadata.java:114)
    at org.apache.spark.sql.connect.client.SparkConnectClient$.<init>(SparkConnectClient.scala:329)
    at org.apache.spark.sql.connect.client.SparkConnectClient$.<clinit>(SparkConnectClient.scala)
    at org.apache.spark.sql.application.ConnectRepl$.doMain(ConnectRepl.scala:61)
    at org.apache.spark.sql.application.ConnectRepl$.main(ConnectRepl.scala:50)
    at org.apache.spark.sql.application.ConnectRepl.main(ConnectRepl.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at coursier.bootstrap.launcher.a.a(Unknown Source)
    at coursier.bootstrap.launcher.Launcher.main(Unknown Source)
Caused by: java.lang.ClassNotFoundException: org.sparkproject.connect.client.com.google.common.io.BaseEncoding
    at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    ... 12 more

---------------------------
 
Do you have any idea why this is happening and how to solve it?
Thank you.