Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-3570

Fix Spark smoke test failures

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0
    • spark, tests
    • None

    Description

      Currently, Spark smoke test doesn't work for the following reasons.
      (e.g., https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/244/COMPONENTS=hdfs.yarn.spark@spark,OS=centos-7-x86_64-deploy/console)

      1. Specifying yarn-client or yarn-cluster for the --master parameter is no longer supported. The master URL and the deploy mode should be separately specified using --master and --deploy-mode respectively.

      2. JavaSparkSQLExample has a bug in Spark 3.0.1. https://issues.apache.org/jira/browse/SPARK-34759

      Attachments

        Activity

          People

            sekikn Kengo Seki
            sekikn Kengo Seki
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 0.5h
                0.5h