[SPARK-32121] ExternalShuffleBlockResolverSuite failed on Windows

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0, 3.0.1
• Fix Version/s: 3.0.1, 3.1.0
• Component/s: Tests

Environment

Windows 10

• Assignee: Cheng Pan
• Reporter: Cheng Pan
• Created: 28/Jun/20 19:27
• Updated: 12/Dec/22 18:11
• Resolved: 02/Jul/20 10:23

Description

The method

ExecutorDiskUtils.createNormalizedInternedPathname

should consider the Windows file separator.

[ERROR] Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.132 s <<< FAILURE! - in org.apache.spark.network.shuffle.ExternalShuffleBlockResolverSuite
[ERROR] testNormalizeAndInternPathname(org.apache.spark.network.shuffle.ExternalShuffleBlockResolverSuite)  Time elapsed: 0 s  <<< FAILURE!
org.junit.ComparisonFailure: expected:</foo[/bar/]baz> but was:</foo[\bar\]baz>
        at org.apache.spark.network.shuffle.ExternalShuffleBlockResolverSuite.assertPathsMatch(ExternalShuffleBlockResolverSuite.java:160)
        at org.apache.spark.network.shuffle.ExternalShuffleBlockResolverSuite.testNormalizeAndInternPathname(ExternalShuffleBlockResolverSuite.java:149)

Issue Links

• relates to: SPARK-32149 Improve file path name normalisation at block resolution within the external shuffle service
• links to: [Github] Pull Request #28940 (pan3793)