[FLINK-19598] E2E test Kerberized YARN application on Docker test (default input) fails on CI

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.12.0
• Component/s: Build System / Azure Pipelines
• Assignee: Unassigned
• Reporter: Zhu Zhu
• Created: 13/Oct/20 06:00
• Updated: 09/Nov/20 15:49
• Resolved: 09/Nov/20 15:49

Description

The root cause might be that the HDFS NameNode failed to start:

2020-10-12T14:03:45.7653530Z 20/10/12 13:57:51 ERROR namenode.NameNode: Failed to start namenode.
2020-10-12T14:03:45.7653932Z java.net.BindException: Port in use: 0.0.0.0:50470
2020-10-12T14:03:45.7654346Z 	at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:998)
2020-10-12T14:03:45.7655069Z 	at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:935)
2020-10-12T14:03:45.7655596Z 	at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:171)
2020-10-12T14:03:45.7656143Z 	at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:842)
2020-10-12T14:03:45.7656665Z 	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:693)
2020-10-12T14:03:45.7657159Z 	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:906)
2020-10-12T14:03:45.7657651Z 	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:885)
2020-10-12T14:03:45.7658151Z 	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1626)
2020-10-12T14:03:45.7658661Z 	at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1694)
2020-10-12T14:03:45.7659086Z Caused by: java.net.BindException: Address already in use
2020-10-12T14:03:45.7659418Z 	at sun.nio.ch.Net.bind0(Native Method)
2020-10-12T14:03:45.7659700Z 	at sun.nio.ch.Net.bind(Net.java:433)
2020-10-12T14:03:45.7660012Z 	at sun.nio.ch.Net.bind(Net.java:425)
2020-10-12T14:03:45.7660397Z 	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:220)
2020-10-12T14:03:45.7660872Z 	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
2020-10-12T14:03:45.7661348Z 	at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
2020-10-12T14:03:45.7661954Z 	at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:993)
2020-10-12T14:03:45.7662289Z 	... 8 more

https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_apis/build/builds/7436/logs/127

Issue Links

• is duplicated by: FLINK-18117 "Kerberized YARN per-job on Docker test" fails with "Could not start hadoop cluster."