[FLINK-24550] Can not access job information from a standby jobmanager UI

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.14.0
• Fix Version/s: 1.14.3, 1.15.0
• Component/s: Runtime / Coordination, Runtime / Web Frontend
• Labels: pull-request-available
• Assignee: Chesnay Schepler
• Reporter: Dawid Wysakowicz
• Created: 14/Oct/21 13:48
• Updated: 15/Feb/22 14:29
• Resolved: 05/Nov/21 13:43

Description

One can not access the "running jobs" section (if a job is running) or if the job is completed it can not access the job page. Moreover the overview section does not work in the standby manager if a job is running. The active jobmanager UI works just fine.

2021-10-14 15:45:11,483 ERROR org.apache.flink.runtime.rest.handler.job.JobExceptionsHandler [] - Unhandled exception.
java.util.concurrent.CancellationException: null
        at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2263) ~[?:1.8.0_231]
        at org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCache.java:98) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInfo(DefaultExecutionGraphCache.java:67) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:81) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
        at java.util.Optional.ifPresent(Optional.java:159) [?:1.8.0_231]
        at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:238) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:71) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [flink-dist_2.12-1.14.0.jar:1.14.0]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231]

It seems to be working just fine in 1.13.2

Reported in the ML: https://lists.apache.org/thread.html/r69646f1c943846ed07f9ff80232c8d0cea31222191354871f914484c%40%3Cuser.flink.apache.org%3E

Issue Links

• is duplicated by: FLINK-24496 Querying job details in the REST API fails
• relates to: FLINK-24706 AkkaInvocationHandler silently ignores deserialization errors
• links to: GitHub Pull Request #17608