Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-11104

Revisit computeMinScalarColumnMemReservation for ORC async IO

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Frontend
    • None

    Description

      HdfsScanNode.computeMinScalarColumnMemReservation has estimate to reduce memory reservation for a column lower than DEFAULT_COLUMN_SCAN_RANGE_RESERVATION (4MB).
      https://github.com/apache/impala/blob/df528fe/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java#L2226-L2235

       

      This estimate is based on Parquet table. We need to revisit this estimate for ORC table.

      Attachments

        Activity

          People

            Unassigned Unassigned
            rizaon Riza Suminto
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: