Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-1702

Mesos executor won't start because of a ClassNotFoundException

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Cannot Reproduce
    • 1.0.0
    • None
    • Mesos

    Description

      Some discussion here: http://apache-spark-user-list.1001560.n3.nabble.com/java-lang-ClassNotFoundException-spark-on-mesos-td3510.html

      Fix here (which is probably not the right fix): https://github.com/apache/spark/pull/620

      This was broken in v0.9.0, was fixed in v0.9.1 and is now broken again.

      Error in Mesos executor stderr:

      WARNING: Logging before InitGoogleLogging() is written to STDERR
      I0502 17:31:42.672224 14688 exec.cpp:131] Version: 0.18.0
      I0502 17:31:42.674959 14707 exec.cpp:205] Executor registered on slave 20140501-182306-16842879-5050-10155-0
      14/05/02 17:31:42 INFO MesosExecutorBackend: Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
      14/05/02 17:31:42 INFO MesosExecutorBackend: Registered with Mesos as executor ID 20140501-182306-16842879-5050-10155-0
      14/05/02 17:31:43 INFO SecurityManager: Changing view acls to: vagrant
      14/05/02 17:31:43 INFO SecurityManager: SecurityManager, is authentication enabled: false are ui acls enabled: false users with view permissions: Set(vagrant)
      14/05/02 17:31:43 INFO Slf4jLogger: Slf4jLogger started
      14/05/02 17:31:43 INFO Remoting: Starting remoting
      14/05/02 17:31:43 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://spark@localhost:50843]
      14/05/02 17:31:43 INFO Remoting: Remoting now listens on addresses: [akka.tcp://spark@localhost:50843]
      java.lang.ClassNotFoundException: org/apache/spark/serializer/JavaSerializer
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:270)
      at org.apache.spark.SparkEnv$.instantiateClass$1(SparkEnv.scala:165)
      at org.apache.spark.SparkEnv$.create(SparkEnv.scala:176)
      at org.apache.spark.executor.Executor.<init>(Executor.scala:106)
      at org.apache.spark.executor.MesosExecutorBackend.registered(MesosExecutorBackend.scala:56)
      Exception in thread "Thread-0" I0502 17:31:43.710039 14707 exec.cpp:412] Deactivating the executor libprocess

      The problem is that it can't find the class.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            bouk Bouke van der Bijl
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment