Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-15341

[Go] ipc.Reader Record Array leak

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • None
    • Go
    • None

    Description

      ipc.Reader produces records whose columns have a reference count of two. As a result, once the reference count of Records produced by the Reader goes to zero, the column arrays will still have a reference count of one and won't be properly accounted for.

      The loading of the columns in the record is driven from this loop:
      https://github.com/influxdata/idpe/blob/185d09d8250c456e6b8417fde622f921b8850c06/vendor/github.com/apache/arrow/go/arrow/ipc/file_reader.go#L338

      The memory for the array buffers is allocated here:
      https://github.com/influxdata/idpe/blob/185d09d8250c456e6b8417fde622f921b8850c06/vendor/github.com/apache/arrow/go/arrow/ipc/file_reader.go#L361

      ctx.loadArray ultimately calls array.MakeFromData which returns an array with a reference count already set to one.

      Finally, array.NewRecord retains each of its columns:
      https://github.com/influxdata/idpe/blob/cd8adab2c1d512f68376064cb7089067d4df567f/vendor/github.com/apache/arrow/go/arrow/array/record.go#L156

      it seems like the code in ipc.Reader that constructs the column arrays needs to release them once the Record has ownership.

      Attachments

        Activity

          People

            Unassigned Unassigned
            cwolff Christopher Wolff
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: