Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6025

User code ClassLoader not used when KryoSerializer fallbacks to serialization for copying

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      Reported in ML: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/AWS-exception-serialization-problem-td12063.html

      This is caused by a known Kryo issue with its JavaSerializer: https://github.com/EsotericSoftware/kryo/pull/483.

      It happens when a Throwable is to be copied by the KryoSerialzer. Since we use the JavaSerializer for throwables, and JavaSerializer doesn't support copying, the KryoSerializer fallbacks to use de-/serialization for the throwable. The problem is that on deserialization, the classloader that the ObjectInputStream uses may be overriden, and doesn't specifically uses Kryo's configured classloader (i.e., the user code class loader), and results in ClassNotFoundException.

      Generally, this may happen if the user also registers to use the JavaSerializer for their types.

      To fix the problem for Throwable serializing in the KryoSerializer, we could either consider registering our own fixed JavaSerializer for throwables, or wait for the Kryo fix to be released (to be fixed in Kryo 4.0.1 release).

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            tzulitai Tzu-Li (Gordon) Tai
            tzulitai Tzu-Li (Gordon) Tai
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment