Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-5517

Provide size-aware set operations in value vectors

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.11.0
    • Fix Version/s: 1.11.0
    • Component/s: None
    • Labels:

      Description

      DRILL-5211 describes a memory fragmentation issue in Drill. The resolution is to limit vector sizes to 16 MB (the size of Netty memory allocation "slabs.") Effort starts by providing "size-aware" set operations in value vectors which:

      • Operate as setSafe() while vectors are below 16 MB.
      • Throw a new, specific exception (VectorOverflowException) if setting the value (and growing the vector) would exceed the vector limit.

      The methods in value vectors then become the foundation on which we can construct size-aware record batch "writers."

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                paul-rogers Paul Rogers
                Reporter:
                paul-rogers Paul Rogers
                Reviewer:
                Karthikeyan Manivannan
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: