Uploaded image for project: 'TinkerPop'
  1. TinkerPop
  2. TINKERPOP-1426

GryoSerializer should implement Java serialization interface

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.1
    • Fix Version/s: 3.3.0
    • Component/s: io
    • Labels:
      None

      Description

      There is description for Serializer in spark:

      • Implementations of this trait should implement:
        *
      • 1. a zero-arg constructor or a constructor that accepts a [[org.apache.spark.SparkConf]]
      • as parameter. If both constructors are defined, the latter takes precedence.
        *
      • 2. Java serialization interface.

      Class GryoSerializer in Tinkerepop extends Serializer, but does not implement java.io.Serializable.
      It works well before Spark 2.0. But with Spark 2.0, it changed by SPARK-13926 for Dependency,scala.
      Gyro and all its fields must implement Java serialisation interface, otherwise hundreds of test cases are failed as:

      Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task not serializable: java.io.NotSerializableException: org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer
      Serialization stack:

      • object not serializable (class: org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer, value: org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer@1b12ec8e)
      • field (class: org.apache.spark.ShuffleDependency, name: serializer, type: class org.apache.spark.serializer.Serializer)
      • object (class org.apache.spark.ShuffleDependency, org.apache.spark.ShuffleDependency@7a4f876a)
      • field (class: scala.Tuple2, name: _2, type: class java.lang.Object)
      • object (class scala.Tuple2, (MapPartitionsRDD[1] at mapToPair at InputFormatRDD.java:46,org.apache.spark.ShuffleDependency@7a4f876a))

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                okram Marko A. Rodriguez
                Reporter:
                qihanycx Chen Xin Yu
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: