Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6025

User code ClassLoader not used when KryoSerializer fallbacks to serialization for copying

    Details

      Description

      Reported in ML: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/AWS-exception-serialization-problem-td12063.html

      This is caused by a known Kryo issue with its JavaSerializer: https://github.com/EsotericSoftware/kryo/pull/483.

      It happens when a Throwable is to be copied by the KryoSerialzer. Since we use the JavaSerializer for throwables, and JavaSerializer doesn't support copying, the KryoSerializer fallbacks to use de-/serialization for the throwable. The problem is that on deserialization, the classloader that the ObjectInputStream uses may be overriden, and doesn't specifically uses Kryo's configured classloader (i.e., the user code class loader), and results in ClassNotFoundException.

      Generally, this may happen if the user also registers to use the JavaSerializer for their types.

      To fix the problem for Throwable serializing in the KryoSerializer, we could either consider registering our own fixed JavaSerializer for throwables, or wait for the Kryo fix to be released (to be fixed in Kryo 4.0.1 release).

        Attachments

          Activity

            People

            • Assignee:
              tzulitai Tzu-Li (Gordon) Tai
              Reporter:
              tzulitai Tzu-Li (Gordon) Tai
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: