Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-78

Provide a ByteBuffer based write path

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • parquet-mr
    • None

    Description

      Similar to the effort to use a ByteBuffer in the read path, we should provide a path to use a ByteBuffer in the write path.

      The proposed idea is to provide an allocator in the writer that is passed down to CapacityByteArrayOutputStream and is used to allocate a ByteBuffer instead of the byte array that it currently uses.

      I've attached a patch based on the 1.5 release that does that. There are two implementations of the allocator - one for Heap memory and one for Direct memory. In addition, the allocator interface allows for a 'release' method so that reference counted memory allocators like Netty's ByteBuf can be used to provide an allocator.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              parthc Parth Chandra
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: