Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-1050

Avro PHP consumes too much memory due to code in io.php (AvroStringIO append_str)

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.6.0, 1.6.1, 1.6.2, 1.6.3
    • 1.7.0
    • php
    • Fix memory growth issue with Avro PHP due to code in AvroStringIO.append_str.

    Description

      While attempting to encode large messages, our team found that the PHP run-time would exceed its memory limit and die. Profiling and use of memory_get_usage showed us that we were spending a lot of time in append_str and that the PHP process grew after every call to this method, resulting in the leak-like behavior that caused the process to grow and then crash. We rewrote append_str to use the string concatenation operator instead of the somewhat elaborate scheme in place today (str_split, followed by the copying of one byte at a time from the latter array to the final string) and the memory usage issues disappeared with no loss of functionality. We would like to provide the patch to fix this issue.

      Attachments

        1. AVRO-1050-2.diff
          2 kB
          A B
        2. AVRO-1050.diff
          0.7 kB
          A B

        Activity

          People

            abawany A B
            abawany A B
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: