Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7127 Fetch-on-demand metadata for the impalad-side catalog
  3. IMPALA-7128

Extract interfaces for frontend interaction with catalog objects

    XMLWordPrintableJSON

    Details

    • Epic Color:
      ghx-label-4

      Description

      Currently both the catalogd and the impalad use the same implementation classes for catalog objects such as Table, Function, HdfsPartition, etc. Those implementations are fairly tightly coupled to the current design of the catalog.

      For the new catalog design, it will be easier to have a new set of classes rather than attempting to reuse or modify the existing ones for a few reasons:

      • the existing ones are mutable in the catalogd and thus have to worry quite a bit about synchronization, whereas the new design uses immutable catalog objects
      • the existing ones assume "all or nothing" loading whereas the new design involves lazy-loading relevant bits of info
      • we don't want to destabilize the existing implementation, so we want to avoid any nontrivial modifications of the existing code.

      Extracting a set of interfaces such as 'FeTable', 'FeDb', etc, will help decouple the frontend from the particular catalog implementation.

        Attachments

          Activity

            People

            • Assignee:
              tlipcon Todd Lipcon
              Reporter:
              tlipcon Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: