Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8949

Remove references to preferredNodeLocalityData in javadoc and print warning when used

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 1.5.0
    • Spark Core, YARN
    • None

    Description

      The SparkContext constructor that takes preferredNodeLocalityData has not worked since before Spark 1.0. Also, the feature in SPARK-4352 is strictly better than a correct implementation of that feature.

      We should remove any documentation references to that feature and print a warning when it is used saying it doesn't work.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-2089 With YARN, preferredNodeLocalityData isn't honored

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue SPARK-12627 Remove SparkContext preferred locations constructor

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          links to

          Pull request #7874 [Github] Pull Request #7874 (darkjh)

          Activity

            People

              darkjh Han JU
              pwendell Patrick Wendell
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: