Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17828

Remove obsolete generate-changelist.py


    • Type: Documentation
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.0.2
    • Fix Version/s: 2.1.0
    • Component/s: Build
    • Labels:


      In Spark 1.x releases we provided the CHANGES.txt file which proved useful to know exactly the differences and commit IDs between releases

      In Spark 2.x we made packaging changes (moving make-distribution.sh to be under dev and different structure for the important Spark classes e.g. no big assembly jar), so I expect work wasn't done to ensure the changes file was still being provided

      Note that users can view the release notes online (https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315420&version=12336857) so I don't know if there is merit in including this file in the distribution, but I've had a look in the codebase and noticed the script used to generate said file is still present in master and definitely out of date:
      https://github.com/apache/spark/blob/master/dev/create-release/generate-changelist.py, here we see the last commit is from the 27th of August 2014

      SPARK_HOME = os.environ["SPARK_HOME"]
      NEW_RELEASE_VERSION = "1.0.0"
      PREV_RELEASE_GIT_TAG = "v0.9.1"

      If we change this to

      SPARK_HOME = os.environ["SPARK_HOME"]
      NEW_RELEASE_VERSION = "2.0.1"
      PREV_RELEASE_GIT_TAG = "v2.0.0"

      we can create a CHANGES.txt.new file

      So I think this file should be either be removed in 2.x. or updated across our currently maintained branches and then we'd need to run the make-distribution script to ensure the file is picked up and included in the distribution

      I see no JIRAs mentioning "changes" under the spark project and no discussion of this on either mailing list. I downloaded the Spark 2.0.0 tgz (no 2.0.1 tgz at http://spark.apache.org/downloads.html) and verified there is no changes file here.




            • Assignee:
              aroberts Adam Roberts
              aroberts Adam Roberts
            • Votes:
              0 Vote for this issue
              3 Start watching this issue


              • Created: