Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-3372

Building Spark fails on Debian 9 and Ubuntu 16.04

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.5.0
    • spark
    • None

    Description

      I ran ./gradlew spark-pkg on Debian 9 and Ubuntu 16.04 and came across the following error:

      + /usr/bin/R CMD check --as-cran --no-tests SparkR_2.4.5.tar.gz
      * using log directory ‘/home/vagrant/bigtop/output/spark/spark-core-2.4.5/R/SparkR.Rcheck’
      * using R version 3.3.3 (2017-03-06)
      * using platform: x86_64-pc-linux-gnu (64-bit)
      * using session charset: UTF-8
      * using options ‘--no-tests --as-cran’
      * checking for file ‘SparkR/DESCRIPTION’ ... OK
      * checking extension type ... Package
      * this is package ‘SparkR’ version ‘2.4.5’
      * checking CRAN incoming feasibility ...Error in .check_package_CRAN_incoming(pkgdir) : 
        dims [product 24] do not match the length of object [0]
      Execution halted
      debian/rules:33: recipe for target 'override_dh_auto_build' failed
      make[1]: *** [override_dh_auto_build] Error 1
      make[1]: Leaving directory '/home/vagrant/bigtop/output/spark/spark-core-2.4.5'
      debian/rules:30: recipe for target 'binary' failed
      make: *** [binary] Error 2
      dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2
      debuild: fatal error at line 1116:
      dpkg-buildpackage -rfakeroot -us -uc -b
       failed
      
      > Task :spark-deb FAILED
      
      FAILURE: Build failed with an exception.
      

      This error has already been reported as SPARK-24152.

      As commented in that issue, R prior to 3.4 seems to cause this problem. Debian 9 and Ubuntu 16.04 uses 3.3.3 and 3.2.3 respectively, and this error doesn't occur at least on CentOS 7 (R 3.6.0) and Debian 10 (R 3.5.2).

      I think we can disable the --as-cran option temporally as mentioned here and bring it back when we drop Debian 9 and Ubuntu 16.04 support, because we're not going to publish any artifact to CRAN.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            sekikn Kengo Seki
            sekikn Kengo Seki
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 50m
                50m

                Slack

                  Issue deployment