Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-4736

[Go] Optimize memory usage for CSV writer

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Go
    • None

    Description

      perhaps not for this PR, but, depending on the number of rows and cols this record contains, this may be a very large allocation, and very big memory chunk.
      it could be more interesting performance wise to write n rows instead of everything in one big chunk.

      also, to reduce the memory pressure on the GC, we should probably (re)use this slice-of-slice of strings.

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            anson627 Anson Qian
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: