Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16408

Bind user code class loader to lifetime of a slot

    XMLWordPrintableJSON

Details

    • Hide
      The user code class loader is being reused by the `TaskExecutor` as long as there is at least a single slot allocated for the respective job. This changes Flink's recovery behaviour slightly so that it will not reload static fields. The benefit is that this change drastically reduces pressure on the JVM's metaspace.
      Show
      The user code class loader is being reused by the `TaskExecutor` as long as there is at least a single slot allocated for the respective job. This changes Flink's recovery behaviour slightly so that it will not reload static fields. The benefit is that this change drastically reduces pressure on the JVM's metaspace.

    Description

      In order to avoid class leaks due to creating multiple user code class loaders and loading class multiple times in a recovery case, I would suggest to bind the lifetime of a user code class loader to the lifetime of a slot. More precisely, the user code class loader should live at most as long as the slot which is using it.

      Attachments

        1. Metaspace-OOM.png
          39 kB
          Echo Lee

        Issue Links

          Activity

            People

              trohrmann Till Rohrmann
              trohrmann Till Rohrmann
              Votes:
              0 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: