Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16408

Bind user code class loader to lifetime of a slot

    XMLWordPrintableJSON

    Details

    • Release Note:
      Hide
      The user code class loader is being reused by the `TaskExecutor` as long as there is at least a single slot allocated for the respective job. This changes Flink's recovery behaviour slightly so that it will not reload static fields. The benefit is that this change drastically reduces pressure on the JVM's metaspace.
      Show
      The user code class loader is being reused by the `TaskExecutor` as long as there is at least a single slot allocated for the respective job. This changes Flink's recovery behaviour slightly so that it will not reload static fields. The benefit is that this change drastically reduces pressure on the JVM's metaspace.

      Description

      In order to avoid class leaks due to creating multiple user code class loaders and loading class multiple times in a recovery case, I would suggest to bind the lifetime of a user code class loader to the lifetime of a slot. More precisely, the user code class loader should live at most as long as the slot which is using it.

        Attachments

        1. Metaspace-OOM.png
          39 kB
          Echo Lee

          Issue Links

            Activity

              People

              • Assignee:
                trohrmann Till Rohrmann
                Reporter:
                trohrmann Till Rohrmann
              • Votes:
                0 Vote for this issue
                Watchers:
                14 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: