Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-1109

wrong API docs for pyspark map function

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.9.0
    • 0.9.1, 1.0.0
    • PySpark
    • None

    Description

      The source code/API docs for the pyspark RDD map function says:

      def map(self, f, preservesPartitioning=False):
      """
      Return a new RDD containing the distinct elements in this RDD.
      """
      def func(split, iterator): return imap(f, iterator)
      return PipelinedRDD(self, func, preservesPartitioning)

      I think that was incorrectly cut-and-pasted from the distinct() function, and should actually say "Return a new RDD by applying a function to each element of this RDD."

      Attachments

        Activity

          People

            prashant Prashant Sharma
            dcarroll@cloudera.com Diana Carroll
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: