Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-47965

Avoid orNull in TypedConfigBuilder and OptionalConfigEntry

    XMLWordPrintableJSON

Details

    Description

      Configuration values/keys cannot be nulls. We should fix:

      diff --git a/core/src/main/scala/org/apache/spark/internal/config/ConfigBuilder.scala b/core/src/main/scala/org/apache/spark/internal/config/ConfigBuilder.scala
      index 1f19e9444d38..d06535722625 100644
      --- a/core/src/main/scala/org/apache/spark/internal/config/ConfigBuilder.scala
      +++ b/core/src/main/scala/org/apache/spark/internal/config/ConfigBuilder.scala
      @@ -94,7 +94,7 @@ private[spark] class TypedConfigBuilder[T](
         import ConfigHelpers._
      
         def this(parent: ConfigBuilder, converter: String => T) = {
      -    this(parent, converter, Option(_).map(_.toString).orNull)
      +    this(parent, converter, { v: T => v.toString })
         }
      
         /** Apply a transformation to the user-provided values of the config entry. */
      

      Attachments

        Activity

          People

            gurwls223 Hyukjin Kwon
            gurwls223 Hyukjin Kwon
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: