[FLINK-21555] HA does not work when using mesos scheduler

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.12.0
• Component/s: Deployment / Mesos
• Labels: auto-deprioritized-major auto-unassigned
• Assignee: Unassigned
• Reporter: Qinghui Xu
• Created: 02/Mar/21 09:35
• Updated: 25/Jun/21 07:00
• Resolved: 25/Jun/21 07:00

Description

When trying to start a flink session cluster in mesos, it's crashing because of jackson dependency issue:

org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to initialize the cluster entrypoint MesosSessionClusterEntrypoint. at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:191) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:529) [flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.mesos.entrypoint.MesosSessionClusterEntrypoint.main(MesosSessionClusterEntrypoint.java:103) [flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002]Caused by: java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper; at com.amazonaws.partitions.PartitionsLoader.<clinit>(PartitionsLoader.java:54) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at com.amazonaws.regions.RegionMetadataFactory.create(RegionMetadataFactory.java:30) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at com.amazonaws.regions.RegionUtils.initialize(RegionUtils.java:64) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at com.amazonaws.regions.RegionUtils.getRegionMetadata(RegionUtils.java:52) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at com.amazonaws.regions.EndpointToRegion.guessRegionOrRegionNameForEndpoint(EndpointToRegion.java:101) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at com.amazonaws.regions.EndpointToRegion.guessRegionNameForEndpoint(EndpointToRegion.java:41) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at com.amazonaws.regions.EndpointToRegion.guessRegionNameForEndpointWithDefault(EndpointToRegion.java:48) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at com.amazonaws.AmazonWebServiceClient.computeSignerByURI(AmazonWebServiceClient.java:429) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at com.amazonaws.AmazonWebServiceClient.setEndpoint(AmazonWebServiceClient.java:318) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at com.amazonaws.services.s3.AmazonS3Client.setEndpoint(AmazonS3Client.java:728) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at com.facebook.presto.hive.s3.PrestoS3FileSystem.createAmazonS3Client(PrestoS3FileSystem.java:661) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at com.facebook.presto.hive.s3.PrestoS3FileSystem.initialize(PrestoS3FileSystem.java:216) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.fs.s3.common.AbstractS3FileSystemFactory.create(AbstractS3FileSystemFactory.java:126) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:468) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:389) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.core.fs.Path.getFileSystem(Path.java:292) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.runtime.blob.BlobUtils.createFileSystemBlobStore(BlobUtils.java:100) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.runtime.blob.BlobUtils.createBlobStoreFromConfig(BlobUtils.java:89) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:117) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.createHaServices(ClusterEntrypoint.java:309) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:272) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.mesos.entrypoint.MesosSessionClusterEntrypoint.initializeServices(MesosSessionClusterEntrypoint.java:56) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:212) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:173) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_275] at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_275] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682) ~[flink-s3-fs-presto-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:172) ~[flink-dist_2.11-1.12-criteo-1605205002.jar:1.12-criteo-1605205002] ... 2 more