Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7127 Fetch-on-demand metadata for the impalad-side catalog
  3. IMPALA-7536

Re-evaluate Avro/Kudu/HBase table performance with fetch-from-catalogd

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Catalog
    • Labels:
    • Epic Color:
      ghx-label-6

      Description

      The Avro, Kudu,and HBase table support for LocalCatalog was written with "fetch-from-source" in mind. Thus, they spend considerable effort to determine the schema on the impalad side when the table is loaded. In "fetch from catalogd" mode, the catalogd has already done this schema resolution, and we're likely better off not doing the extra work on the impalad. We should do a pass back over those table types to make sure there isn't unnecessary work being performed.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              tlipcon Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: