Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-4333

[C++] Sketch out design for kernels and "query" execution in compute layer

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Done
    • None
    • 6.0.0
    • C++

    Description

      It would be good to formalize the design of kernels and the controlling query execution layer (e.g. volcano batch model?) to understand the following:

      Contracts for kernels:

      • Thread safety of kernels?
      • When Kernels should allocate memory vs expect preallocated memory?  How to communicate requirements for a kernels memory allocaiton?
      • How to communicate the whether a kernels execution is parallelizable across a ChunkedArray?  How to determine if the order to execution across a ChunkedArray is important?
      • How to communicate when it is safe to re-use the same buffers and input and output to the same kernel?

      What does the threading model look like for the higher level of control?  Where should synchronization happen?

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              emkornfield@gmail.com Micah Kornfield
              Votes:
              2 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: