Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-26973

Kubernetes version support strategy on test nodes / backend

    XMLWordPrintableJSON

Details

    • Test
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.0.0
    • None
    • Kubernetes, Spark Core
    • None

    Description

      Kubernetes has a policy for supporting three minor releases and the current ones are defined here: https://github.com/kubernetes/sig-release/blob/master/releases/patch-releases.md.

      Moving from release 1.x to 1.(x+1) happens roughly every 100 days:https://gravitational.com/blog/kubernetes-release-cycle.

      This has an effect on dependencies upgrade at the Spark on K8s backend and the version of Minikube required to be supported for testing. One other issue is what the users actually want at the given time of a release. Some popular vendors like EKS(https://aws.amazon.com/eks/faqs/) have their own roadmap for releases and may not catch up fast (what is our view on this).

      Follow the comments for a recent discussion on the topic: https://github.com/apache/spark/pull/23814.

      Clearly we need a strategy for this.

      A couple of options for the current state of things:

      a) Support only the last two versions, but that leaves out a version that still receives patches.

      b) Support only the latest, which makes testing easier, but leaves out other currently maintained version.

      A good strategy will optimize at least the following:

      1) percentage of users satisfied at release time.

      2) how long it takes to support the latest K8s version

      3) testing requirements eg. minikube versions used

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            skonto Stavros Kontopoulos
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: