Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-4533

The lambda expression is not being transferred to the executors

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.8.2
    • None
    • Interpreters
    • None

    Description

      I am running Zeppelin with spark 2.2.4 on master=yarn & deploy-mode = client

      I found any lambda expression is not being transferred to the executors

      e.g. ```val rdd = sc.parallelize(Seq(3, 4))
      rdd.map(n => n+1).count```

      the error message:

      ```

      java.lang.ClassNotFoundException: $line56182880833.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$anonfun$1

      ```

      And here is the run time information

      Attachments

        Activity

          People

            Unassigned Unassigned
            maofeng Mao FENG
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: