Details
-
Bug
-
Status: Closed
-
Blocker
-
Resolution: Fixed
-
None
Description
Due to RFC-46 the profile of the data being serialized by Hudi had changed considerably: previously we're mostly passing around Avro payloads, while now we hold our own internal HoodieRecord implementations.
When classes are not explicitly registered w/ Kryo, it would have to serialize class fully qualified name (FQN) as id every time an object is serialized, which carries a lot of unnecessary overhead.
To work this around in #7026 added HoodieSparkKryoRegistrar registering some of the commonly serialized Hudi classes. However, during rebasing/merging of the RFC-46 feature branch this changes have been partially reverted and so we need to restore that.
Attachments
Issue Links
- links to