Critical Description
Original message here.
First, an observation. If you check https://repository.apache.org/content/groups/snapshots-group/org/apache/karaf/webconsole/org.apache.karaf.webconsole.http/4.2.0-SNAPSHOT/maven-metadata.xml, you'll see this metadata declares latest SNAPSHOT version to be 4.2.0-20170713.142530-162.
But if you try to actually fetch this version (that's what Aether is doing), you'll get HTTP 404, because there's no such version - there's 4.2.0-20170713.142529-162 - 1 second difference, but very important - this SNAPSHOT is not resolvable.
I started wondering about Nexus problems, about maven-deploy-plugin bugs, but then I run this under debugger. I didn't notice this problem when using Maven 3.5.0, but it failed under 3.3.9 (that's the version used by Jenkins).
Another worrying sign was this log:
[INFO] --- maven-deploy-plugin:2.8.2:deploy (default-deploy) @ org.apache.karaf.features.core --- ... [INFO] Downloading: http://localhost:8081/nexus/content/repositories/snapshots/org/apache/karaf/features/org.apache.karaf.features.core/maven-metadata.xml [INFO] Downloaded: http://localhost:8081/nexus/content/repositories/snapshots/org/apache/karaf/features/org.apache.karaf.features.core/maven-metadata.xml (315 B at 4.6 KB/sec) [INFO] Downloading: http://localhost:8081/nexus/content/repositories/snapshots/org/apache/karaf/features/org.apache.karaf.features.core/maven-metadata.xml [INFO] Downloaded: http://localhost:8081/nexus/content/repositories/snapshots/org/apache/karaf/features/org.apache.karaf.features.core/maven-metadata.xml (315 B at 10.6 KB/sec) [INFO] Uploading: http://localhost:8081/nexus/content/repositories/snapshots/org/apache/karaf/features/org.apache.karaf.features.core/4.2.0-SNAPSHOT/maven-metadata.xml [INFO] Uploaded: http://localhost:8081/nexus/content/repositories/snapshots/org/apache/karaf/features/org.apache.karaf.features.core/4.2.0-SNAPSHOT/maven-metadata.xml (805 B at 31.4 KB/sec) [INFO] Uploading: http://localhost:8081/nexus/content/repositories/snapshots/org/apache/karaf/features/org.apache.karaf.features.core/4.2.0-SNAPSHOT/maven-metadata.xml [INFO] Uploaded: http://localhost:8081/nexus/content/repositories/snapshots/org/apache/karaf/features/org.apache.karaf.features.core/4.2.0-SNAPSHOT/maven-metadata.xml (805 B at 30.2 KB/sec) ...
It looked as duplicate metadata upload. I was fooled by reading maven-deploy-plugin source code, because it actually may upload metadata multiple times (in case of attached artifacts).
But then, under debugger I saw this:
generators = {java.util.ArrayList@9874} size = 4
0 = {org.apache.maven.repository.internal.RemoteSnapshotMetadataGenerator@9881}
snapshots: java.util.Map = {java.util.LinkedHashMap@9890} size = 0
legacyFormat: boolean = false
1 = {org.apache.maven.repository.internal.RemoteSnapshotMetadataGenerator@9882}
snapshots: java.util.Map = {java.util.LinkedHashMap@9889} size = 0
legacyFormat: boolean = false
2 = {org.apache.maven.repository.internal.VersionsMetadataGenerator@9883}
versions: java.util.Map = {java.util.LinkedHashMap@9887} size = 0
processedVersions: java.util.Map = {java.util.LinkedHashMap@9888} size = 0
3 = {org.apache.maven.repository.internal.VersionsMetadataGenerator@9884}
versions: java.util.Map = {java.util.LinkedHashMap@9885} size = 0
processedVersions: java.util.Map = {java.util.LinkedHashMap@9886} size = 0
which means that org.eclipse.aether.internal.impl.DefaultDeployer#getMetadataGenerators() returned/used too many generators. And I found that this is the root problem - we have two instances of RemoteSnapshotMetadataGenerator and both may transform "SNAPSHOT" version to two different timestamped versions - then artifact is uploaded once (with first timestamped version) and then metadata is uploaded twice - first time with correct version and then with 1-second later version...
Why? Why duplicate generators? The problem is class realm of ... org.apache.karaf.tooling:karaf-services-maven-plugin when running on Maven 3.3.9.
maven-aether-provider-3.3.9.jar declares (META-INF/plexus/components.xml):
<component> <role>org.eclipse.aether.impl.MetadataGeneratorFactory</role> <role-hint>snapshot</role-hint> <implementation>org.apache.maven.repository.internal.SnapshotMetadataGeneratorFactory</implementation> <description /> <isolated-realm>false</isolated-realm> </component>
and maven-resolver-provider-3.5.0.jar (direct dependency of karaf-services-maven-plugin) has (META-INF/plexus/components.xml):
<component> <role>org.eclipse.aether.impl.MetadataGeneratorFactory</role> <role-hint>snapshot</role-hint> <implementation>org.apache.maven.repository.internal.SnapshotMetadataGeneratorFactory</implementation> <description /> <isolated-realm>false</isolated-realm> </component>
Even if these declaration from different deps lead to single Class of org.apache.maven.repository.internal.SnapshotMetadataGeneratorFactory (from maven-aether-provider-3.3.9.jar), we still have TWO instances of such class.