Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-29081

Replace calls to SerializationUtils.clone on properties with a faster implementation.

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      I don't have empirical evidence, but due to the sheer number of times I have seen spark in the SerializationUtils.clone method in stack traces I've taken I figured it could use some love.

      This functionality was implemented to fix a bug where we were accidentally modifying the properties man used by other threads. That solution was to use SerializationUtils.clone, but that is not necessary. I suggest just creating a new Properties object and iterating over the key-value pairs in the original map and adding them one by one.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                databricks-david-lewis David Lewis
                Reporter:
                databricks-david-lewis David Lewis
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 24h
                  24h
                  Remaining:
                  Remaining Estimate - 24h
                  24h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified