Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
3.5.0
Description
The configuration "spark.eventLog.buffer.kb" has a definition with KiB unit.
private[spark] val EVENT_LOG_OUTPUT_BUFFER_SIZE = ConfigBuilder("spark.eventLog.buffer.kb") .doc("Buffer size to use when writing to output streams, in KiB unless otherwise specified.") .version("1.0.0") .bytesConf(ByteUnit.KiB) .createWithDefaultString("100k")
It is used in a context acting as the size of bytes.
protected val outputBufferSize = sparkConf.get(EVENT_LOG_OUTPUT_BUFFER_SIZE).toInt
This may be a mistake and needs to be fixed.
The default buffer size for the event-log output buffer will be 100k as configured.
Attachments
Issue Links
- links to