Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23732

Broken link to scala source code in Spark Scala api Scaladoc

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Trivial
    • Resolution: Fixed
    • 2.3.0, 2.3.1
    • 2.1.3, 2.2.2, 2.3.2, 2.4.0

    Description

      Scala source code link in Spark api scaladoc is broken.

      Turns out instead of the relative path to the scala files the "€{FILE_PATH}.scala" expression in https://github.com/apache/spark/blob/master/project/SparkBuild.scala is generating the absolute path from the developers computer. In this case, if I try to access the source link on https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.Accumulable, it tries to take me to https://github.com/apache/spark/tree/v2.3.0/Users/sameera/dev/spark/core/src/main/scala/org/apache/spark/Accumulable.scala

      where "/Users/sameera/dev/spark" portion of the URL is coming from the developers macos home folder.

      There seems to be no change in the code responsible for generating this path during the build in /project/SparkBuild.scala :

      Line # 252:

      scalacOptions in Compile ++= Seq(
      s"-target:jvm-${scalacJVMVersion.value}",
      "-sourcepath", (baseDirectory in ThisBuild).value.getAbsolutePath // Required for relative source links in scaladoc
      ),
      

      Line # 726

      // Use GitHub repository for Scaladoc source links
      unidocSourceBase := s"https://github.com/apache/spark/tree/v${version.value}",
      
      scalacOptions in (ScalaUnidoc, unidoc) ++= Seq(
      "-groups", // Group similar methods together based on the @group annotation.
      "-skip-packages", "org.apache.hadoop"
      ) ++ (
      // Add links to sources when generating Scaladoc for a non-snapshot release
      if (!isSnapshot.value) {
      Opts.doc.sourceUrl(unidocSourceBase.value + "€{FILE_PATH}.scala")
      } else {
      Seq()
      }
      )

       

      It seems more like a developers dev environment issue.

      I was successfully able to reproduce this in my dev environment. Environment details attached. 

       

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            vanzin Marcelo Masiero Vanzin
            yogeshtewari Yogesh Tewari
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment