Affects Version/s: None
Fix Version/s: None
Component/s: COMP - Compiler
Sometimes, we would like to be able to query system status. For example:
1. Disk space.
2. Number of components of a dataset.
3. Memory usage.
And many others. Being able to query such information and utilize the power of the query language and the runtime makes a great investigative/diagnostic tool.
Currently, there is no easy way to do that. Such functionality can be achieved through:
1. External datasets but that takes a lot of work in terms of development and usage.
2. Use specific diagnostic end points but then that is also a lot of development work and you end up losing the ability to use the query language.
Current proposal is to introduce datasource functions. A datasource function is different from normal functions as:
1. Takes constants ( as opposed to variables).
2. Has location constraints "For a start, it can be on all nodes".
An example would be the function dataset_resources(String dataverse, String dataset);
This function takes a dataverse and a dataset and produce a set of json representing the disk resources of the dataset.