Export - CSV (All fields)
Export - CSV (Current fields)
SPARK-38204
All state operators are at a risk of inconsistency between state partitioning and operator partitioning
uploadFileUri should preserve file scheme
pyspark hangs in local mode running rdd map operation
Stream-stream outer join has a possible correctness issue due to weakly read consistent on outer iterators
Disable lineSep option in 'from_csv' and 'schema_of_csv'
The accumulator of ArrayAggregate should copy the intermediate result if string, struct, array, or map
Add PyArrow, pandas and plotly to release Docker image dependencies
Invalid LocalMergedBlockData cause task hang
UnsafeHashedRelation should serialize numKeys out
Upgrade to Py4J 0.10.9.5
Arbitrary shell command injection via Utils.unpack()
Spark tasks stuck forever due to deadlock between TaskMemoryManager and UnsafeExternalSorter
Add IMMEDIATE statement to the DB2 dialect truncate implementation
Spark Shuffle Index Cache calculates memory usage wrong
KMeans fails with NegativeArraySizeException for K = 50000 after issue #27758 was introducedSPARK-36808
Upgrade Kafka to 2.8.1
Exponential planning time in case of non-deterministic function
test_reuse_worker_of_parallelize_range is flaky
sequence over dates with month interval is producing incorrect results
when charVarcharAsString is true, char datatype partition table query incorrect
Support predicate pushdown and column pruning for de-duped CTEsSPARK-37675
Prevent overwriting of push shuffle merged files once the shuffle is finalized
Recursive view `df` detected (cycle: `df` -> `df`)
plot.hist throws AttributeError on pandas=1.3.5
Add appId interface to KubernetesConf
Spark should not dedup the groupingExpressions when the first child of Union has duplicate columns
Set file.encoding to UTF-8 for SBT tests
Analyzer can fail when join left side and right side are the same view
Need to update Partition URI after renaming table in InMemoryCatalog
Upgrade ORC to 1.6.13
TPCDS 1TB q72 fails when spark.sql.optimizer.dynamicPartitionPruning.reuseBroadcastOnly is false
Update K8s doc to recommend K8s 1.20+
AQE can change bhj to smj if no extra shuffle introduce
Fix ColumnVectorUtils.populate to handle CalendarIntervalType correctly
ExecutorMonitor.timedOutExecutors should be deterministic
ExecutorMonitor.onExecutorRemoved should handle ExecutorDecommission as finished
Support docker-desktop K8S integration test in SBT
Query with cast containing non-nullable columns fails with AQE on Spark 3.1.1
Encoder cannot be found when a tuple component is a type alias for an Array
SPARK-38045
More strict validation on plan check for stream-stream join unit test
Fix KafkaSource/KafkaMicroBatch flaky test due to non-deterministic timing
Add IntegrationTestBackend.describePods to support all K8s test backends
Structured streaming not working in history server when using LevelDB
Support K8s namespace parameter in SBT K8s IT
Support K8s imageTag parameter in SBT K8s IT
Update atexit function to avoid issues with late binding
Hive script transform with order by and limit will return fake rows- SPARK-38080
Flaky test: StreamingQueryManagerSuite: 'awaitAnyTermination with timeout and resetTerminated'
Support cloud-backend in K8s IT with SBT- SPARK-38084
Support `SKIP_PYTHON` and `SKIP_R` in `run-tests.py`