Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-2984

Ruby: Unnecessary memory allocations during serialization

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.10.0
    • Fix Version/s: 1.11.0, 1.10.2
    • Component/s: ruby
    • Labels:
      None

      Description

      Profiling our Ruby event publishing pipeline, we see two big sources of unnecessary memory allocations:

      1. Avro::IO:DatumWriter#write_data allocates a hash of constant validation options
      2. Avro::SchemaValidator::Result allocates an errors array which is unnecessary for the common case of no validation errors

      This is on a very hot codepath for us and accounts for roughly 30% of the memory allocated by this part of the event publishing pipeline.

      Github PR coming shortly.

        Attachments

          Activity

            People

            • Assignee:
              joelturkel Joel Turkel
              Reporter:
              joelturkel Joel Turkel
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: