Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Not A Bug
-
None
-
None
-
None
-
None
Description
Today Drill provides the drill-override.conf file to set Drill properties, and the drill-env.sh file to provide custom launch properties. Today, most users seem to have a copy of DRILL_HOME per node, and thus they copy these two files per-node. The result is that the two files act as both the overall "site" configuration (for all nodes) and the "per-node" configuration for that one node.
In addition, some distributions of Drill (such as MapR), modify the "user" config files with settings for that distribution. Now, the same files hold settings for the distribution, site and node.
The approach works, but is awkward. Ideally, provide the option to have three sets of files: for the distribution, site, and node.
The proposal is to extend configuration to provide additional levels:
- Drill defaults (drill default and module conf files, code in drill-config.sh)
- Distribution settings (special JVM settings, say)
- Site settings (standard log or spill file locations)
- Node settings
- Launch settings (environment variables, -Dname=value options)
The improvement becomes more important if a user employs NFS, MapR FS or YARN to automatically deploy the site-wide files. In that case, the site files cannot also act as per-node files.
The improvement also simplifies upgrades. Today, users must copy customizations from and old to a new install. With the revision, Drill files are complely separated from user files, making upgrades (of software) easier.
For backward compatibility, the site and node directories are optional and ignored if the environment variables are not set. The site and node config files should be optional: skip them if they do not exist.