Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-3372

Building Spark fails on Debian 9 and Ubuntu 16.04

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5.0
    • Component/s: spark
    • Labels:
      None

      Description

      I ran ./gradlew spark-pkg on Debian 9 and Ubuntu 16.04 and came across the following error:

      + /usr/bin/R CMD check --as-cran --no-tests SparkR_2.4.5.tar.gz
      * using log directory ‘/home/vagrant/bigtop/output/spark/spark-core-2.4.5/R/SparkR.Rcheck’
      * using R version 3.3.3 (2017-03-06)
      * using platform: x86_64-pc-linux-gnu (64-bit)
      * using session charset: UTF-8
      * using options ‘--no-tests --as-cran’
      * checking for file ‘SparkR/DESCRIPTION’ ... OK
      * checking extension type ... Package
      * this is package ‘SparkR’ version ‘2.4.5’
      * checking CRAN incoming feasibility ...Error in .check_package_CRAN_incoming(pkgdir) : 
        dims [product 24] do not match the length of object [0]
      Execution halted
      debian/rules:33: recipe for target 'override_dh_auto_build' failed
      make[1]: *** [override_dh_auto_build] Error 1
      make[1]: Leaving directory '/home/vagrant/bigtop/output/spark/spark-core-2.4.5'
      debian/rules:30: recipe for target 'binary' failed
      make: *** [binary] Error 2
      dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2
      debuild: fatal error at line 1116:
      dpkg-buildpackage -rfakeroot -us -uc -b
       failed
      
      > Task :spark-deb FAILED
      
      FAILURE: Build failed with an exception.
      

      This error has already been reported as SPARK-24152.

      As commented in that issue, R prior to 3.4 seems to cause this problem. Debian 9 and Ubuntu 16.04 uses 3.3.3 and 3.2.3 respectively, and this error doesn't occur at least on CentOS 7 (R 3.6.0) and Debian 10 (R 3.5.2).

      I think we can disable the --as-cran option temporally as mentioned here and bring it back when we drop Debian 9 and Ubuntu 16.04 support, because we're not going to publish any artifact to CRAN.

        Attachments

          Activity

            People

            • Assignee:
              sekikn Kengo Seki
              Reporter:
              sekikn Kengo Seki
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 50m
                50m