it would be great to have some kind of test dataset support; currently there is the q_test_init.sql which is quite large; and I'm often override it with an invalid string; because I write independent qtests most of the time - and the load of src and other tables are just a waste of time for me ; not to mention that the loading of those tables may also trigger breakpoints - which is a bit annoying.
Most of the tests are "only" using the src table and possibly 2 others; however the main init script contains a bunch of tables - meanwhile there are quite few other tests which could possibly also benefit from a more general feature; for example the creation of bucket_small is present in 20 q files.
the proposal would be to enable the qfiles to be annotated with metadata like datasets:
proposal for storing a dataset:
- the loader script would be at: data/datasets/_NAME_/load.hive.sql
- the table data could be stored under that location
a draft about this; and other qfiles related ideas:
|qfiles: prepare .q files for using datasets||Resolved|