[FLINK-17266] WorkerResourceSpec is not serializable

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.11.0
• Fix Version/s: 1.11.0
• Component/s: Deployment / Mesos
• Labels: pull-request-available
• Assignee: Till Rohrmann
• Reporter: Gary Yao
• Created: 20/Apr/20 11:36
• Updated: 21/Apr/20 07:16
• Resolved: 21/Apr/20 07:16

Description

MesosResourceManager cannot acquire new resources due to WorkerResourceSpec not being serializable.

Caused by: java.lang.Exception: Could not open output stream for state backend
        at org.apache.flink.runtime.zookeeper.filesystem.FileSystemStateStorageHelper.store(FileSystemStateStorageHelper.java:70) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
        at org.apache.flink.runtime.zookeeper.ZooKeeperStateHandleStore.addAndLock(ZooKeeperStateHandleStore.java:136) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
        at org.apache.flink.mesos.runtime.clusterframework.store.ZooKeeperMesosWorkerStore.putWorker(ZooKeeperMesosWorkerStore.java:216) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
        at org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager.startNewWorker(MesosResourceManager.java:441) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
        ... 34 more
Caused by: java.io.NotSerializableException: org.apache.flink.runtime.resourcemanager.WorkerResourceSpec
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184) ~[?:1.8.0_242]
        at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) ~[?:1.8.0_242]
        at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) ~[?:1.8.0_242]
        at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) ~[?:1.8.0_242]
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) ~[?:1.8.0_242]
        at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) ~[?:1.8.0_242]
        at org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:594) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
        at org.apache.flink.runtime.zookeeper.filesystem.FileSystemStateStorageHelper.store(FileSystemStateStorageHelper.java:62) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
        at org.apache.flink.runtime.zookeeper.ZooKeeperStateHandleStore.addAndLock(ZooKeeperStateHandleStore.java:136) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
        at org.apache.flink.mesos.runtime.clusterframework.store.ZooKeeperMesosWorkerStore.putWorker(ZooKeeperMesosWorkerStore.java:216) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
        at org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager.startNewWorker(MesosResourceManager.java:441) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
        ... 34 more

Issue Links

• is caused by: FLINK-16437 Make SlotManager allocate resource from ResourceManager at the worker granularity.