Major Description
When I used hadoop-gremlin 3.4.6 to export a graph to HDFS, I encountered an error.
My gremlin query looks like:
graph = GraphFactory.open('conf/fdb-psave-export.properties')
graph.compute(SparkGraphComputer).program(CloneVertexProgram.build().create()).submit().get()
org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not find a type identifier for the class : class java.lang.Byte. Make sure the value to serialize has a type identifier registered for its class. at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider._wrapAsIOE(DefaultSerializerProvider.java:509) ~[gremlin-shaded-3.4.6.jar:3.4.6] at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:482) ~[gremlin-shaded-3.4.6.jar:3.4.6] at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[gremlin-shaded-3.4.6.jar:3.4.6] at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3906) ~[gremlin-shaded-3.4.6.jar:3.4.6] at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3177) ~[gremlin-shaded-3.4.6.jar:3.4.6] at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONWriter.writeVertex(GraphSONWriter.java:82) ~[gremlin-core-3.4.6.jar:3.4.6] at org.apache.tinkerpop.gremlin.hadoop.structure.io.graphson.GraphSONRecordWriter.write(GraphSONRecordWriter.java:72) ~[hadoop-gremlin-3.4.6.jar:3.4.6] at org.apache.tinkerpop.gremlin.hadoop.structure.io.graphson.GraphSONRecordWriter.write(GraphSONRecordWriter.java:42) ~[hadoop-gremlin-3.4.6.jar:3.4.6] at org.apache.spark.internal.io.HadoopMapReduceWriteConfigUtil.write(SparkHadoopWriter.scala:356) ~[spark-core_2.11-2.4.0.jar:2.4.0] at org.apache.spark.internal.io.SparkHadoopWriter$$anonfun$4.apply(SparkHadoopWriter.scala:130) ~[spark-core_2.11-2.4.0.jar:2.4.0] at org.apache.spark.internal.io.SparkHadoopWriter$$anonfun$4.apply(SparkHadoopWriter.scala:127) ~[spark-core_2.11-2.4.0.jar:2.4.0] at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1394) [spark-core_2.11-2.4.0.jar:2.4.0] at org.apache.spark.internal.io.SparkHadoopWriter$.org$apache$spark$internal$io$SparkHadoopWriter$$executeTask(SparkHadoopWriter.scala:139) [spark-core_2.11-2.4.0.jar:2.4.0] at org.apache.spark.internal.io.SparkHadoopWriter$$anonfun$3.apply(SparkHadoopWriter.scala:83) [spark-core_2.11-2.4.0.jar:2.4.0] at org.apache.spark.internal.io.SparkHadoopWriter$$anonfun$3.apply(SparkHadoopWriter.scala:78) [spark-core_2.11-2.4.0.jar:2.4.0] at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) [spark-core_2.11-2.4.0.jar:2.4.0] at org.apache.spark.scheduler.Task.run(Task.scala:121) [spark-core_2.11-2.4.0.jar:2.4.0] at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:416) [spark-core_2.11-2.4.0.jar:2.4.0] at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360) [spark-core_2.11-2.4.0.jar:2.4.0] at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:422) [spark-core_2.11-2.4.0.jar:2.4.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_202] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_202] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_202]
It looks like the default GraphSONRecordWriter/GraphSONRecordReader does not support the serialization/deserialization of java.lang.Byte. But according to https://tinkerpop.apache.org/docs/3.4.6/dev/io/#_extended_2, adding the custom module GraphSONXModuleV2d0/GraphSONXModuleV3d0 can fix this. The experient with a quick fix proves the fix is working.