|
|
|
SPARK-47934
|
SPARK-45869
Ensure trailing slashes in `HistoryServer` URL redirections
|
huangzhir
|
huangzhir
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-47894
|
SPARK-45869
Add `Environment` page to Master UI
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-47446
|
SPARK-45869
Make `BlockManager` warn before `removeBlockInternal`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-47364
|
SPARK-45869
Make `PluginEndpoint` warn when plugins reply for one-way message
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-47215
|
SPARK-45869
Reduce the number of required threads in MasterSuite
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-47207
|
SPARK-45869
Support `spark.driver.timeout` and `DriverTimeoutPlugin`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-47181
|
SPARK-45869
Fix `MasterSuite` to validate the number of registered workers
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-47162
|
SPARK-45869
Exclude `CodeHaus Jackson` dependencies from Master/Worker/HistoryServer classpaths
|
Unassigned
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-47030
|
SPARK-45869
Add `WebBrowserTest`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-47020
|
SPARK-45869
Fix `RealBrowserUISeleniumSuite`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-47000
|
SPARK-45869
Use `getTotalMemorySize` in `WorkerArguments`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46997
|
SPARK-45869
Enable `spark.worker.cleanup.enabled` by default
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46967
|
SPARK-45869
Hide `Thread Dump` and `Heap Histogram` of `Dead` executors in `Executors` UI
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46965
|
SPARK-45869
Check logType in Utils.getLog
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46924
|
SPARK-45869
Fix `Load New` button in `Master/HistoryServer` Log UI
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46907
|
SPARK-45869
Show driver log location in Spark History Server
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46903
|
SPARK-45869
Support Spark History Server Log UI
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46899
|
SPARK-45869
Remove POST APIs from `MasterWebUI` when spark.ui.killEnabled is false
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46889
|
SPARK-45869
Validate `spark.master.ui.decommission.allow.mode` setting
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46888
|
SPARK-45869
Fix `Master` to reject worker kill request if decommission is disabled
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46883
|
SPARK-45869
Support `/json/clusterutilization` API
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46881
|
SPARK-45869
Support `spark.deploy.workerSelectionPolicy`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46872
|
SPARK-45869
Recover `log-view.js` to be non-module
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46870
|
SPARK-45869
Support Spark Master Log UI
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46869
|
SPARK-45869
Add `logrotate` to Spark docker files
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46868
|
SPARK-45869
Support Spark Worker Log UI
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46846
|
SPARK-45869
Make `WorkerResourceInfo` extend `Serializable` explicitly
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46827
|
SPARK-45869
Make RocksDBPersistenceEngine to support a symbolic link
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46817
|
SPARK-45869
Fix `spark-daemon.sh` usage by adding `decommission` command
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46800
|
SPARK-45869
Support `spark.deploy.spreadOutDrivers`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46799
|
SPARK-45869
Improve `MasterSuite` to use nanoTime-based appIDs and workerIDs
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46797
|
SPARK-45869
Rename spark.deploy.spreadOut to spark.deploy.spreadOutApps
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46749
|
SPARK-45869
Document `SPARK_LOG_*` and `SPARK_PID_DIR`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46748
|
SPARK-45869
Remove `*slave*.sh` scripts
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46704
|
SPARK-45869
Fix `MasterPage` to sort `Running Drivers` table by `Duration` column correctly
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46664
|
SPARK-45869
Improve `Master` to recover quickly in case of zero workers and apps
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46369
|
SPARK-45869
Remove `kill` link from RELAUNCHING drivers in MasterPage
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46368
|
SPARK-45869
Support `readyz` in REST Submission API
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46353
|
SPARK-45869
Refactor to improve `RegisterWorker` unit test
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46348
|
SPARK-45869
Support `spark.deploy.recoveryTimeout`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46346
|
SPARK-45869
Fix Master to update worker from UNKNOWN to ALIVE on RegisterWorker message
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46344
|
SPARK-45869
Warn properly when a driver exists successfully but master is disconnected
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46320
|
SPARK-45869
Support `spark.master.rest.host`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46313
|
SPARK-45869
Log Spark HA recovery duration
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46311
|
SPARK-45869
Log the final state of drivers during `Master.removeDriver`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46301
|
SPARK-45869
Support `spark.worker.(initial|max)RegistrationRetries`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46299
|
SPARK-45869
Make `spark.deploy.recovery*` documentation up-to-date
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46292
|
SPARK-45869
Show a summary of workers in MasterPage
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46258
|
SPARK-45869
Add RocksDBPersistenceEngine
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46217
|
SPARK-45869
Include `Driver/App` data in `PersistenceEngineBenchmark`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46216
|
SPARK-45869
Improve `FileSystemPersistenceEngine` to support compressions
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46215
|
SPARK-45869
Improve `FileSystemPersistenceEngine` to allow non-exist parents
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46205
|
SPARK-45869
Improve `PersistenceEngine` performance with `KryoSerializer`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46193
|
SPARK-45869
Add PersistenceEngineBenchmark
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46191
|
SPARK-45869
Improve `FileSystemPersistenceEngine.persist` error message in case of the existing file
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46095
|
SPARK-45869
Document REST API for Spark Standalone Cluster
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-46012
|
SPARK-45869
EventLogFileReader should not read rolling logs if appStatus is missing
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45961
|
SPARK-45869
Document `spark.master.*` configurations
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45948
|
SPARK-45869
Make single-pod spark jobs respect spark.app.id
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45934
|
SPARK-45869
Fix `Spark Standalone` documentation table layout
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45901
|
SPARK-45869
Collect and update `spark-standalone.md` with new confs
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45867
|
SPARK-45869
Support `spark.worker.idPattern`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45843
|
SPARK-45869
Support `killall` in REST Submission API
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45819
|
SPARK-45869
Support `clear` in REST Submission API
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45785
|
SPARK-45869
Support `spark.deploy.appNumberModulo` to rotate app number
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45777
|
SPARK-45869
Support `spark.test.appId` in `LocalSchedulerBackend`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45774
|
SPARK-45869
Support `spark.master.ui.historyServerUrl` in `ApplicationPage`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45763
|
SPARK-45869
Improve `MasterPage` to show `Resource` column only when it exists
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45756
|
SPARK-45869
Support `spark.master.useAppNameAsAppId.enabled`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45754
|
SPARK-45869
Support `spark.deploy.appIdPattern`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45753
|
SPARK-45869
Support `spark.deploy.driverIdPattern`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45749
|
SPARK-45869
Fix Spark History Server to sort `Duration` column properly
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45500
|
SPARK-45869
Show the number of abnormally completed drivers in MasterPage
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45474
|
SPARK-45869
Support top-level filtering in MasterPage JSON API
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45229
|
SPARK-45869
Show the number of drivers waiting in SUBMITTED status
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45197
|
SPARK-45869
Make StandaloneRestServer add JavaModuleOptions to drivers
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45187
|
SPARK-45869
Fix WorkerPage to use the same pattern for `logPage` urls
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-45174
|
SPARK-45869
Support `spark.deploy.maxDrivers`
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-44857
|
SPARK-45869
Fix getBaseURI error in Spark Worker LogPage UI buttons
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-44214
|
SPARK-45869
Support Spark Driver Live Log UI
|
Dongjoon Hyun
|
Dongjoon Hyun
|
|
Resolved |
Fixed
|
|
|
|
|