[HDFS-12548] HDFS Jenkins build is unstable on branch-2

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.9.0
• Component/s: build
• Target Version/s: 2.10.3
• Assignee: Unassigned
• Reporter: Rushabh Shah
• Created: 26/Sep/17 14:24
• Updated: 16/May/22 15:19

Description

Feel free move the ticket to another project (e.g. infra).
Recently I attached branch-2 patch while working on one jira HDFS-12386
There were at-least 100 failed and timed out tests. I am sure they are not related to my patch.
Also I came across another jira which was just a javadoc related change and there were around 100 failed tests.
Below are the details for pre-commits that failed in branch-2
1 HDFS-12386 attempt 1

Ran on slave: asf912.gq1.ygridcore.net/H12
Failed with following error message:
Build timed out (after 300 minutes). Marking the build as aborted.
Build was aborted
Performing Post build task...

2. HDFS-12386 attempt 2

Ran on slave: asf900.gq1.ygridcore.net
Failed with following error message:
FATAL: command execution failed
Command close created at
	at hudson.remoting.Command.<init>(Command.java:60)
	at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1123)
	at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1121)
	at hudson.remoting.Channel.close(Channel.java:1281)
	at hudson.remoting.Channel.close(Channel.java:1263)
	at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1128)
Caused: hudson.remoting.Channel$OrderlyShutdown
	at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1129)
	at hudson.remoting.Channel$1.handle(Channel.java:527)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:83)
Caused: java.io.IOException: Backing channel 'H0' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:192)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:257)
	at com.sun.proxy.$Proxy125.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1043)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1035)
	at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:155)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:735)
	at hudson.model.Build$BuildExecution.build(Build.java:206)
	at hudson.model.Build$BuildExecution.doRun(Build.java:163)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
	at hudson.model.Run.execute(Run.java:1735)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:405)

3. HDFS-12531 attempt 1

Ran on slave:  asf911.gq1.ygridcore.net
Failed with following error message:
FATAL: command execution failed
Command close created at
	at hudson.remoting.Command.<init>(Command.java:60)
	at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1123)
	at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1121)
	at hudson.remoting.Channel.close(Channel.java:1281)
	at hudson.remoting.Channel.close(Channel.java:1263)
	at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1128)
Caused: hudson.remoting.Channel$OrderlyShutdown
	at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1129)
	at hudson.remoting.Channel$1.handle(Channel.java:527)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:83)
Caused: java.io.IOException: Backing channel 'H11' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:192)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:257)
	at com.sun.proxy.$Proxy125.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1043)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1035)
	at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:155)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:735)
	at hudson.model.Build$BuildExecution.build(Build.java:206)
	at hudson.model.Build$BuildExecution.doRun(Build.java:163)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
	at hudson.model.Run.execute(Run.java:1735)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:405)
Build step 'Execute shell' marked build as failure

The above 3 attempts ran on different slaves so I don't think its a slave related issue.