Description
When using fair scheduler mode for thrift server, we may have unpredictable result.
val pool = sessionToActivePool.get(parentSession.getSessionHandle) if (pool != null) { sqlContext.sparkContext.setLocalProperty(SparkContext.SPARK_SCHEDULER_POOL, pool) }
Here is an example:
We have some query will use default pool, however it submit to 'normal' pool.
I changed code and add some log.Got some strange result.
Then i found out that the localProperties of SparkContext may has unpredictable result when call setLocalProperty. And since thriftserver use thread pool to execute queries, it will trigger this bug sometimes.
/** * Set a local property that affects jobs submitted from this thread, such as the Spark fair * scheduler pool. User-defined properties may also be set here. These properties are propagated * through to worker tasks and can be accessed there via * [[org.apache.spark.TaskContext#getLocalProperty]]. * * These properties are inherited by child threads spawned from this thread. This * may have unexpected consequences when working with thread pools. The standard java * implementation of thread pools have worker threads spawn other worker threads. * As a result, local properties may propagate unpredictably. */ def setLocalProperty(key: String, value: String) { if (value == null) { localProperties.get.remove(key) } else { localProperties.get.setProperty(key, value) } }
Attachments
Attachments
Issue Links
- is duplicated by
-
SPARK-26992 Fix STS scheduler pool correct delivery
- Resolved
- links to