Description
Kryo is still pretty slow because it works on individual objects and relative expensive to allocate. For Exchange operator, because the schema for key and value are already defined, we can create a specialized serializer to handle the specific schemas of key and value.
Attachments
Attachments
Issue Links
- is blocked by
-
SPARK-6367 Use the proper data type for those expressions that are hijacking existing data types.
- Resolved
- is related to
-
SPARK-6986 Makes SparkSqlSerializer2 support sort-based shuffle with sort merge
- Resolved
- links to