Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
4.0.0
-
None
-
None
Description
To address the issue of the Spark website repository size
reaching the storage limit for GitHub-hosted runners [1], I suggest
enhancing step [2] in our release process by relocating the
documentation releases from the dev[3] directory to the release
directory[4]. Then it would captured by the Apache Archives
service[5] to create permanent links, which would be alternative
endpoints for our documentation, like
https://dist.apache.org/repos/dist/dev/spark/v3.5.2-rc5-docs/_site/index.html
for
https://spark.apache.org/docs/3.5.2/index.html
Note that the previous example still uses the staging repository,
which will become
https://archive.apache.org/dist/spark/docs/3.5.2/index.html.
For older releases hosted on the Spark website [6], we also need to
upload them via SVN manually.
After that, when we reach the threshold again, we can delete some of
the old ones on page [6], and update their links on page [7] or use
redirection.
[1] https://lists.apache.org/thread/o0w4gqoks23xztdmjjj26jkp1yyg2bvq
[2] https://spark.apache.org/release-process.html#upload-to-apache-release-directory
[3] https://dist.apache.org/repos/dist/dev/spark/v3.5.2-rc5-docs/
[4] https://dist.apache.org/repos/dist/release/spark/docs/3.5.2
[5] https://archive.apache.org/dist/spark/
[6] https://github.com/apache/spark-website/tree/asf-site/site/docs
[7] https://spark.apache.org/documentation.html