[FLINK-6025] User code ClassLoader not used when KryoSerializer fallbacks to serialization for copying

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1.5, 1.2.1, 1.3.0
• Component/s: API / Type Serialization System
• Assignee: Tzu-Li (Gordon) Tai
• Reporter: Tzu-Li (Gordon) Tai
• Created: 11/Mar/17 10:58
• Updated: 14/Mar/17 10:46
• Resolved: 13/Mar/17 15:55

Description

Reported in ML: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/AWS-exception-serialization-problem-td12063.html

This is caused by a known Kryo issue with its JavaSerializer: https://github.com/EsotericSoftware/kryo/pull/483.

It happens when a Throwable is to be copied by the KryoSerialzer. Since we use the JavaSerializer for throwables, and JavaSerializer doesn't support copying, the KryoSerializer fallbacks to use de-/serialization for the throwable. The problem is that on deserialization, the classloader that the ObjectInputStream uses may be overriden, and doesn't specifically uses Kryo's configured classloader (i.e., the user code class loader), and results in ClassNotFoundException.

Generally, this may happen if the user also registers to use the JavaSerializer for their types.

To fix the problem for Throwable serializing in the KryoSerializer, we could either consider registering our own fixed JavaSerializer for throwables, or wait for the Kryo fix to be released (to be fixed in Kryo 4.0.1 release).

Issue Links

• links to: GitHub Pull Request #3517
• links to: GitHub Pull Request #3518
• links to: GitHub Pull Request #3519