Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.13.0
Description
It would be useful for Drill to have some sort of metastore which would enable Drill to remember previously defined schemata so Drill doesn’t have to do the same work over and over again.
It allows to store schema and statistics, which will allow to accelerate queries validation, planning and execution time. Also it increases stability of Drill and allows to avoid different kind if issues: "schema change Exceptions", "limit 0" optimization and so on.
One of the main candidates is Hive Metastore.
Starting from 3.0 version Hive Metastore can be the separate service from Hive server:
https://cwiki.apache.org/confluence/display/Hive/AdminManual+Metastore+3.0+Administration
Optional enhancement is storing Drill's profiles, UDFs, plugins configs in some kind of metastore as well.
Attachments
Issue Links
- incorporates
-
DRILL-6852 Adapt current Parquet Metadata cache implementation to use Drill Metastore API
- Resolved
-
DRILL-7098 File Metadata Metastore Plugin
- Resolved
- is depended upon by
-
DRILL-5192 REFRESH table METADATA as default system option
- Resolved
- is related to
-
DRILL-7028 Reduce the planning time of queries on large Parquet tables with large metadata cache files
- Resolved
-
DRILL-7430 Drill Metastore analyze improvements
- Open
-
DRILL-7684 Implement Drill Metastore metadata usage for Hive storage plugin
- Open
- relates to
-
DRILL-3588 Write back to Hive Metastore
- Open
-
DRILL-6035 Specify Drill's JSON behavior
- Open
-
DRILL-1328 Support table statistics
- Resolved
- requires
-
DRILL-5603 Replace String file paths to Hadoop Path
- Resolved
-
DRILL-6604 Upgrade Drill Hive client to Hive3.1 version
- Resolved
- links to