Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20607

Add new unit tests to ShuffleSuite

    XMLWordPrintableJSON

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.1.2
    • Fix Version/s: 2.3.0
    • Component/s: Shuffle, Tests
    • Labels:
      None

      Description

      1.adds the new unit tests.
      testing would be performed when there is no shuffle stage,
      shuffle will not generate the data file and the index files.
      2.Modify the 'SPARK-4085 rerun map stage if reduce stage cannot find its local shuffle file' unit test,
      parallelize is 1 but not is 2, Check the index file and delete.

        Attachments

          Activity

            People

            • Assignee:
              heary-cao caoxuewen
              Reporter:
              heary-cao caoxuewen
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: