Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
it would be great to have some kind of test dataset support; currently there is the q_test_init.sql which is quite large; and I'm often override it with an invalid string; because I write independent qtests most of the time - and the load of src and other tables are just a waste of time for me ; not to mention that the loading of those tables may also trigger breakpoints - which is a bit annoying.
Most of the tests are "only" using the src table and possibly 2 others; however the main init script contains a bunch of tables - meanwhile there are quite few other tests which could possibly also benefit from a more general feature; for example the creation of bucket_small is present in 20 q files.
the proposal would be to enable the qfiles to be annotated with metadata like datasets:
--! qt:dataset:src,bucket_small
proposal for storing a dataset:
- the loader script would be at: data/datasets/_NAME_/load.hive.sql
- the table data could be stored under that location
a draft about this; and other qfiles related ideas:
https://docs.google.com/document/d/1KtcIx8ggL9LxDintFuJo8NQuvNWkmtvv_ekbWrTLNGc/edit?usp=sharing
Attachments
Attachments
Issue Links
- Dependent
-
HIVE-19330 multi_insert_partitioned.q fails with "src table does not exist" message.
- Closed
- relates to
-
HIVE-18862 qfiles: prepare .q files for using datasets
- Closed