Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-5466

Can not specify the spark image from the interpreter settings

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 0.9.0
    • 0.11.0, 0.10.1
    • Kubernetes, spark
    • None

    Description

      When Zeppelin launches a spark interpreter pod under k8s mode, it will build the spark submit options through buildSparkSubmitOptions()

       

      But the property "--conf spark.kubernetes.container.image=" will use the value from the "zConf.getK8sSparkContainerImage()". Note that although the interpreter properties overrides the values through

      // interpreter properties overrides the values
      k8sProperties.putAll(Maps.fromProperties(properties));
      

       

      But the spark submit options has already been built. So if user set the "spark.kubernetes.container.image" from the interpreter settings, it will not set the spark image in the spark submit options.

       

      Attachments

        Activity

          People

            rickcheng rickcheng
            rickcheng rickcheng
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 0.5h
                0.5h