Affects Version/s: None
Fix Version/s: None
gobblin supports multiple modes of executions ( CLI, Standalone, cluster-master, cluster-worker, AWS, YARN, MR ) and various command lines utility to run cli and admin commands. The problem is each cli and execution mode has individual script to manage the service, which brings following problems.
Having individual script introduces lot of issues
- all scripts handles gobblin variables, user parameters differently, and its highly inconsistent among various different gobblin scripts, not to mention different features supported by different scripts.
- functionality around start, stop, status checking and handling PID's among lot of other things, varies vastly as per the implementation of the script.
- features like GC & JVM params, log4j file selection, classpath calculation, etc... exists in some gobblin scripts but not all, adding to inconsistent user experience.
- code duplication: all the gobblin scripts share lot of common code to handle params, start, stop services, status checks, pid handling, etc... combining all the scripts into 1 not only makes maintenance easier but also brings clarity and consistency.
- Basically, current 13 different scripts adds confusion to new user on how to use Gobblin or how to use it.
1. there can be one gobblin.sh script to handle all gobblin commands and deployment options as per following signature. NOTE: This
gobblin.sh <command> <params>
gobblin.sh <execution-mode> <start|stop|status>
commands values: admin, cli, statestore-check, statestore-clean, historystore-manager, classpath
service values: standalone, cluster-master, cluster-worker, aws, yarn, mr, service
with above change, following becomes valid command.
2. Also all configurations for each mode needs to be structured and de-duped accordingly to make it clear on which config will be picked up for which execution mode. This would be well defined in command help instructions.
NOTE: this refactoring adds all cli and service commands to gobblin.sh and hence changes the syntax for all commands and services.