Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18466

Missing withFilter method causes errors when using for comprehensions in Scala 2.12

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Won't Fix
    • 2.4.0
    • None
    • Spark Core

    Description

      The fact that the RDD class has a filter method but not a withFilter method results in compiler warnings when using RDDs in for comprehensions. As of Scala 2.12, falling back to use of filter is no longer supported, so for comprehensions that use filters will no longer compile. Semantically, the only difference between withFilter and filter is that withFilter is lazy, and since RDDs are lazy by nature, one can simply be aliased to the other.

      Attachments

        Activity

          People

            Unassigned Unassigned
            reggert1980 Richard W. Eggert II
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified