Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9810

Support Kudu's columnar scan format (Apache Arrow)

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Backend
    • ghx-label-4

    Description

      Starting in Kudu 1.12 Kudu supports a columnar wire format that is Apache Arrow compatible. This format is significantly more efficient and faster than the row based format. Impala should support the columnar format, optimally operating on the arrow/columnar format natively.

      Of course using the columnar format natively is the best end result, but it might be beneficial enough to translate to the row based format client side. The Java client recently added a patch for this https://gerrit.cloudera.org/#/c/15983/ and Kudu could add similar C++ client functionality if needed.

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-2017 Lazy materialization of Parquet columns during query

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          relates to

          New Feature - A new feature of the product, which has yet to be developed. KUDU-2077 Return data in Apache Arrow format

          • Major - Major loss of function.
          • Reopened

          Activity

            People

              Unassigned Unassigned
              granthenke Grant Henke
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: