-
Type:
Sub-task
-
Status: Closed
-
Priority:
Critical
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: spark-branch
-
Component/s: spark
-
Labels:
-
Patch Info:Patch Available
Bags cannot be sent to an RDD, as it produces a SelfSpillBag$MemoryLimits not Serializable exception. This results in an error for almost every operation performed after grouping tuples.
This error is fixed after making transient the protected MemoryLimit memLimit attribute inside org.apache.pig.data.SelfSpillBag, but I do not know the impact of this change.
- blocks
-
PIG-4234 Order By error after Group By in Spark
-
- Closed
-