Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7672

Make metadata type required when reading from / writing into Drill Metastore

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.17.0
    • 1.18.0
    • None

    Description

      Metastore consists of components: TABLES, VIEWS etc (so far only TABLES are implemented). Each component metadata can have types. For examples, TABLES metadata can be of the following types: TABLE, SEGMENT, FILE, ROW_GROUP, PARTITION.
      During initial Metastore implementation when reading from / writing into Metastore, metadata type was indicated in filter expressions.
      For Iceberg Metastore where all data is stored in files this was not this critical, basically when information is retrieved about the table, table folder is queried.
      For other Metastore implementations knowing metadata type can be more critical. For example, RDBMS Metastore would store TABLES metadata in different tables thus knowing which table to query would improve performance rather than trying to query all tables.
      Of course, we could traverse query filter and look for the hints which metadata type is needed but it is much better to know required metadata type beforehand without any extra logic.
      Taking into account that Metastore metadata is queried only in Drill code, developer knows beforehand what he needs to get / update / delete.

      This Jira aims to make metadata type required when reading from / writing into Drill Metastore. This change does not have any affect on the users, just internal code refactoring.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            arina Arina Ielchiieva
            arina Arina Ielchiieva
            Vova Vysotskyi Vova Vysotskyi
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment