Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-36051

Remove automatic update of documentation build in the guides

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.2.0
    • Component/s: docs, PySpark
    • Labels:
      None

      Description

      At https://github.com/apache/spark/tree/master/docs#automatically-rebuilding-api-docs,

      cd "$SPARK_HOME/python/docs"
      find .. -type f -name '*.py' \
      | entr -s 'make html && cp -r _build/html/. ../../docs/api/python'
      

      is documented. However, this doesn't work very well:

      1. It doesn't detect the changes in .rst files. But PySpark internally generates rst files so we can't just simply include it in the detection. Otherwise, it goes to an infinite loop
      2. During PySpark documentation generation, it launches some jobs to generate plot images now. This is broken with entr command, and the job fails. Seems like it's related to how entr creates the process internally.

        Attachments

          Activity

            People

            • Assignee:
              hyukjin.kwon Hyukjin Kwon
              Reporter:
              hyukjin.kwon Hyukjin Kwon
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: