[SPARK-35004] Fix Incorrect assertion of "master/worker web ui available behind front-end reverseProxy" in MasterSuite

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.1.1, 3.2.0
• Fix Version/s: 3.1.2, 3.2.0
• Component/s: Tests
• Assignee: Yang Jie
• Reporter: Yang Jie
• Created: 09/Apr/21 07:53
• Updated: 09/Apr/21 13:20
• Resolved: 09/Apr/21 13:19

Description

https://github.com/apache/spark/blob/bfba7fadd2e65c853971fb2983bdea1c52d1ed7f/core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala#L425

// with LocalCluster, we have masters and workers in the same JVM, each overwriting
// system property spark.ui.proxyBase.
// so we need to manage this property explicitly for test
System.getProperty("spark.ui.proxyBase") should startWith  (s"$reverseProxyUrl/proxy/worker-") // Line 425

Line 425 in MasterSuite is considered as unused expression,  If we merge lines 424 and 425 into one：

System.getProperty("spark.ui.proxyBase") should startWith (s"$reverseProxyUrl/proxy/worker-")

this assertion will fail:

- master/worker web ui available behind front-end reverseProxy *** FAILED ***
  The code passed to eventually never returned normally. Attempted 45 times over 5.091914027 seconds. Last failure message: "http://proxyhost:8080/path/to/spark" did not start with substring "http://proxyhost:8080/path/to/spark/proxy/worker-". (MasterSuite.scala:405)

 

 

 

 

Issue Links

• links to: [Github] Pull Request #32105 (LuciferYang)