Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-2423

Avoid allocating buffer obeject in RecordReader's SkipRecords

    XMLWordPrintableJSON

Details

    Description

      Currently each invocation of SkipRecords() for non-repeated fields will create a brand new buffer object[1]. I think it probably worth keep the buffer object alive and just resize it for each skip, as the buffer is just a bitmap for records.

      [1]https://github.com/apache/arrow/blob/main/cpp/src/parquet/column_reader.cc#L1482

      Attachments

        Issue Links

          Activity

            People

              zjpzlz Jinpeng Zhou
              zjpzlz Jinpeng Zhou
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 7.5h
                  7.5h