Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-1771

Support configurable for DirectByteBufferAllocator from Hadoop Configuration

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 1.10.1
    • None
    • parquet-mr

    Description

      Now in HadoopReadOptions class, we cannot change default allocator from Hadoop Configuration.

       

      Add a config `parquet.allocator.direct.enabled` to enable `DirectByteBufferAllocator` in `ParuqetFileReader`. 

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. PARQUET-1006 ColumnChunkPageWriter uses only heap memory.

          • Major - Major loss of function.
          • Open
          links to

          Web Link GitHub Pull Request #749

          Activity

            People

              Unassigned Unassigned
              lishuming ShuMing Li
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: