Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
None
-
None
Description
The function of IndexedKey and PigNullableWritable is similar.
The difference between these two is IndexedKey contains Index,key while PigNullableWritable contains index,key,value.
Besides,the comparators for PigNullableWritable have lot of conditions for the different data types taken care of and IndexedKey can miss some of that. We can try to replace IndexedKey with PigNullableWritable.
rohini: I tried to replace IndexedKey wih PigBNullableWritable but it failed because PigNullableWritable is not serializable. So i will remain IndexedKey in spark package. Can you give me some suggestion?
exception info
Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0.0 in stage 32.0 mpl.io.NullableTuple Serialization stack: - object not serializable (class: org.apache.pig.impl.io.NullableTuple, value: Null: false in [