Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-3540

Fix toolchain to abort if R installation failed

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0
    • build, spark
    • None

    Description

      Building Spark is failing on CentOS 7 due to the lack of the knitr library.
      https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/701/COMPONENTS=spark,OS=centos-7/console

      Error in loadVignetteBuilder(pkgdir, TRUE) : 
      * checking DESCRIPTION meta-information ... OK
      
      
      RPM build errors:
        vignette builder 'knitr' not found
      Execution halted
      error: Bad exit status from /var/tmp/rpm-tmp.PfHp35 (%build)
          Bad exit status from /var/tmp/rpm-tmp.PfHp35 (%build)
      
      > Task :spark-rpm FAILED
      

      The Docker image in question was built without any errors:
      https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/OS=centos-7/167/console

      Notice: /Stage[main]/Bigtop_toolchain::Renv/Package[pandoc]/ensure: created
      Notice: /Stage[main]/Bigtop_toolchain::Renv/Package[R-devel]/ensure: created
      Notice: /Stage[main]/Bigtop_toolchain::Renv/Package[R]/ensure: created
      Notice: /Stage[main]/Bigtop_toolchain::Renv/Exec[install_r_packages]/returns: executed successfully
      Notice: /Stage[main]/Bigtop_toolchain::Protobuf/File[/usr/src/0001-Backport-atomic-operations-with-support-of-arm64-and.patch]/ensure: defined content as '{md5}1c62663f215f577fa07a22f5860eef53'
      Notice: /Stage[main]/Bigtop_toolchain::Protobuf/Exec[download protobuf]/returns: executed successfully
      Notice: /Stage[main]/Bigtop_toolchain::Protobuf/Exec[install protobuf]/returns: executed successfully
      Notice: /Stage[last]/Bigtop_toolchain::Cleanup/Exec[clean packages]/returns: executed successfully
      Notice: /Stage[last]/Bigtop_toolchain::Cleanup/Exec[remove archives]/returns: executed successfully
      Notice: Finished catalog run in 716.17 seconds
      
      ...
      
      BUILD SUCCESSFUL in 17s
      1 actionable task: 1 executed
      Removing intermediate container 7f0ae430745e
       ---> 5a2c5cd62ee4
      Successfully built 5a2c5cd62ee4
      Successfully tagged bigtop/slaves:trunk-centos-7
      + rm -f Dockerfile
      
      BUILD SUCCESSFUL in 13m 18s
      1 actionable task: 1 executed
      + docker push bigtop/slaves:trunk-centos-7
      
      ...
      
      trunk-centos-7: digest: sha256:e8ee97d9d799e0b206b0ffbdc1a846ca19d242d3b762ba7b50099ce6c062fd8e size: 1796
      Finished: SUCCESS
      

      But it actually doesn't have knitr for some reason.

      $ docker pull bigtop/slaves:trunk-centos-7
      trunk-centos-7: Pulling from bigtop/slaves
      Digest: sha256:e8ee97d9d799e0b206b0ffbdc1a846ca19d242d3b762ba7b50099ce6c062fd8e
      Status: Image is up to date for bigtop/slaves:trunk-centos-7
      docker.io/bigtop/slaves:trunk-centos-7
      $ docker run -it bigtop/slaves:trunk-centos-7 Rscript -e 'library(knitr)'
      Error in library(knitr) : there is no package called 'knitr'
      Execution halted
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            sekikn Kengo Seki
            sekikn Kengo Seki
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h 50m
                1h 50m

                Slack

                  Issue deployment