Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-49587

Improve UDF Packet Serialization

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 4.0.0
    • None
    • Connect
    • None

    Description

      When looking at serialized UDF packets it is noticeable that agnostic encoders are taking up quite a bit of space. In many cases they capture quite a bit more than expected (a lot of the objects capture things in their class hierarchy.

      Since we need to be backwards compatible here, we can't just change serialization. Instead the proposal would be to use Serialization proxies for UdfPacket, and perhaps the encoders themselves.

      Attachments

        Activity

          People

            Unassigned Unassigned
            hvanhovell Herman van Hövell
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: