Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19696

Wrong Documentation for Java Word Count Example

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.1.0
    • Component/s: Documentation
    • Labels:
      None

      Description

      Java Word Count example of http://spark.apache.org/examples.html page is defined incorrectly

      '''
      JavaRDD<String> textFile = sc.textFile("hdfs://...");
      JavaRDD<String> words = textFile.flatMap(s -> Arrays.asList(s.split(" ")).iterator())
      .mapToPair(word -> new Tuple2<>(word, 1))
      .reduceByKey((a, b) -> a + b);
      counts.saveAsTextFile("hdfs://...");
      ''

      It should be

      '''
      JavaRDD<String> textFile = sc.textFile("hdfs://...");
      JavaPairRDD<String, Integer> counts = textFile.flatMap(s -> Arrays.asList(s.split(" ")).iterator())
      .mapToPair(word -> new Tuple2<>(word, 1))
      .reduceByKey((a, b) -> a + b);
      counts.saveAsTextFile("hdfs://...");
      ''''

        Attachments

          Activity

            People

            • Assignee:
              srowen Sean Owen
              Reporter:
              gaurav.gopi123 gaurav gupta
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: