[ANY23-370] Jenkins: IllegalStateException: checksum mismatch

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3
• Fix Version/s: 2.5
• Component/s: CIS (Jenkins)
• Assignee: Unassigned
• Reporter: Hans Brende
• Created: 19/Jul/18 03:33
• Updated: 27/Aug/21 15:30
• Resolved: 22/Sep/20 04:47

Description

Jenkins builds (e.g., #1597) have been sporadically timing out after 120 minutes during the archival process, even though the actual build process completes successfully in less than half an hour. I believe that this is due to the following error:

ERROR: Failed to archive \{ 
org.apache.any23/apache-any23-service/2.3-SNAPSHOT/apache-any23-service-2.3-SNAPSHOT.pom=pom.xml, 
org.apache.any23/apache-any23-service/2.3-20180718.234115-48/apache-any23-service-2.3-20180718.234115-48.war=target/apache-any23-service-2.3-SNAPSHOT.war, 
org.apache.any23/apache-any23-service/2.3-20180718.234115-48/apache-any23-service-2.3-20180718.234115-48-without-deps.war=target/apache-any23-service-2.3-SNAPSHOT-without-deps.war, 
org.apache.any23/apache-any23-service/2.3-20180718.234115-48/apache-any23-service-2.3-20180718.234115-48-with-deps.tar.gz=target/apache-any23-service-2.3-SNAPSHOT-with-deps.tar.gz, 
org.apache.any23/apache-any23-service/2.3-20180718.234115-48/apache-any23-service-2.3-20180718.234115-48-with-deps.zip=target/apache-any23-service-2.3-SNAPSHOT-with-deps.zip, 
org.apache.any23/apache-any23-service/2.3-20180718.234115-48/apache-any23-service-2.3-20180718.234115-48-without-deps.tar.gz=target/apache-any23-service-2.3-SNAPSHOT-without-deps.tar.gz, 
org.apache.any23/apache-any23-service/2.3-20180718.234115-48/apache-any23-service-2.3-20180718.234115-48-without-deps.zip=target/apache-any23-service-2.3-SNAPSHOT-without-deps.zip, 
org.apache.any23/apache-any23-service/2.3-20180718.234115-48/apache-any23-service-2.3-20180718.234115-48-server-embedded.tar.gz=target/apache-any23-service-2.3-SNAPSHOT-server-embedded.tar.gz, 
org.apache.any23/apache-any23-service/2.3-20180718.234115-48/apache-any23-service-2.3-20180718.234115-48-server-embedded.zip=target/apache-any23-service-2.3-SNAPSHOT-server-embedded.zip
} due to internal error; falling back to full archiving 

java.lang.IllegalStateException: End of stream while reading number
 at jsync.protocol.BaseReader.readLong(BaseReader.java:40)
 at jsync.protocol.BaseReader.readInt(BaseReader.java:26)
 at jsync.protocol.ChangeStreamReader.next(ChangeStreamReader.java:54)
 at jsync.protocol.ChangeInputStream.next(ChangeInputStream.java:27)
 at jsync.protocol.ChangeInputStream.read(ChangeInputStream.java:71)
 at com.cloudbees.jenkins.plugins.jsync.archiver.MD5DigestingInputStream.read(MD5DigestingInputStream.java:39)
 at com.google.common.io.LimitInputStream.read(LimitInputStream.java:79)
 at java.io.FilterInputStream.read(FilterInputStream.java:107)
 at com.google.common.io.ByteStreams.copy(ByteStreams.java:193)
 at jsync.protocol.FileSequenceReader.read(FileSequenceReader.java:35)
 at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:158)
 at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:76)
 at hudson.maven.MavenBuild$ProxyImpl.performArchiving(MavenBuild.java:512)
 at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:881)
 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
 at hudson.model.Run.execute(Run.java:1794)
 at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
 at hudson.model.ResourceController.execute(ResourceController.java:97)
 at hudson.model.Executor.run(Executor.java:429)

Then, at some point during the "full archiving" process, we get the following message:

Build timed out (after 120 minutes). Marking the build as aborted.

In order to mitigate this timeout error, I've temporarily increased the timeout to 180 minutes. But this should not be necessary: we should try to fix the underlying issue.

Here's another similar error message from build #1599:

ERROR: Failed to archive {...} due to internal error; falling back to full archiving
java.lang.IllegalStateException: checksum mismatch after transfer (900817822 vs. 1339188126); 
/x1/jenkins/jenkins-home/jobs/Any23-trunk/modules/org.apache.any23$apache-any23-service/builds/1599/archive/org.apache.any23/apache-any23-service/2.3-20180719.041833-49/
apache-any23-service-2.3-20180719.041833-49-server-embedded.zip may be corrupted
	at jsync.protocol.FileSequenceReader.read(FileSequenceReader.java:45)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:158)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:76)
	at hudson.maven.MavenBuild$ProxyImpl.performArchiving(MavenBuild.java:512)
	at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:881)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
	at hudson.model.Run.execute(Run.java:1794)
	at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)