Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1429

Apache build of Bigtop test artifacts fails to find correct Hbase jars



    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 0.7.0
    • 0.8.0
    • build, hbase, tests
    • None


      Our testartifacts build fails on Apache CI system with the following error:

      [ERROR] Failed to execute goal on project hbase-smoke: Could not resolve dependencies for project org.apache.bigtop.itest:hbase-smoke:jar:0.8.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.hbase:hbase-common:jar:0.98.2, org.apache.hbase:hbase-common:jar:tests:0.98.2, org.apache.hbase:hbase-server:jar:0.98.2, org.apache.hbase:hbase-server:jar:tests:0.98.2: Could not find artifact org.apache.hbase:hbase-common:jar:0.98.2 in central (http://repo.maven.apache.org/maven2) -> [Help 1]

      Here's the problem:

      • our packages are created on a separate Bigtop CI system that creates and locally installs HBase artifacts built against a particular Hadoop2 version. The resulting artifacts version is - in the current BOM - 0.98.2.
      • the testartifacts build in question is ran on a Apache CI machine, as it needs to publish Bigtop test artifacts to ASF maven repo (Bigtop CI machine doesn't have correct credentials for the deployment, e.g. it isn't allowed to)
      • ASF CI machine doesn't have HBase 0.98.2 artifact locally installed, and tries to fetch them from repo.maven.apache.org. However, HBase releases are produced for both hadoop1 and hadoop2. To make this distinction the artifacts versions are alternated to 0.98.2-hadoop1, and 0.98.2-hadoop2 respectively. As the result, testartifacts build fails being unable to resolve the depedencies.

      There're a few ways to solve the issue:

      1. provision ASF CI hosts beforehand with .m2 cache from bigtop CI machine
      2. deploy artifacts from Bigtop package build phase into 3rd party Maven repo, which can be later used in testartifacts build to resolve the dependencies
      3. finally, alternate Bigtop Hbase package build to produce 0.98.2-hadoop2 artifact. This version will be resolved during testartifacts and the deploy will happen. However, during the cluster tests (at Bigtop CI slaves) correct version from local .m2 caches will be used, hence guaranteeing consistency of the stack


        Issue Links



              cos Konstantin I Boudnik
              cos Konstantin I Boudnik
              0 Vote for this issue
              3 Start watching this issue