Details
-
Sub-task
-
Status: Done
-
Major
-
Resolution: Done
-
0.3.0
-
None
-
Centos7
Description
Multiple bug fixes and recommended improvements were found in the course of implementing METRON-608 that are unrelated to METRON-609 (singlenode install). This jira provides the work items for merging those changes into the mainline Mpack. Almost all items relate to Elasticsearch.
About half the changes impact the Ambari database, and should therefore be done with an Mpack version bump and database upgrade script. These have been moved to METRON-865, and removed from the below description.
Not impacting Ambari database:
- ES pid_dir specification and usage:
- Currently pid_dir is multiply specified in elastic-env.xml and params.py. The config parameter should not be over-ridden in params.py.
- PID_DIR failed to be included in /etc/sysconfig/elasticsearch. It needs to be added to the template in elastic-sysconfig, as it must be provided to ES at launch-time (else the default directory will be used).
- pid_file is specified in params.py, but is not used anywhere. (The ES internal launcher synthesizes it from PID_DIR, and this is appropriate.)
- JAVA_HOME needs to be provided in /etc/sysconfig/elasticsearch (templated in elastic-sysconfig.xml). Its absence causes Centos7 systemctl to fail the ES launch, unless /bin/java is defined (which it isn't necessarily).
- Also in the /etc/sysconfig/elasticsearch template in elastic-sysconfig.xml, the value of ES_JAVA_OPTS incorrectly spans 3 lines. The lines must be terminated with backslashes to effectively become a single line. The current inclusion of newlines in the long string value is acceptable (although unusual) in shellscript, but not in a systemd EnvironmentFile. /etc/sysconfig/elasticsearch must function as both.
- Also in ES_JAVA_OPTS, the two instances of log_dir needs to be followed by a slash '/'
- Recommend adding QuickLinks for Elasticsearch. This is a low-impact change that is very helpful to validate ES health.
- ES health status
- ES indexes list
- In elastic.py, when directories are being pre-created and permissions set, the directory $CONF_DIR/scripts should also be pre-created. I intermittently hit permissions issues with this directory being created later by root, and not properly assigned to elastic_user.
- In several places in elastic.py, "params.elastic_user" is incorrectly used when "params.elastic_group" should be used.
- Undefined "format()" method is used in elastic.py, unnecessarily in File(format("/etc/sysconfig/elasticsearch")...
- Undefined "format()" method is similarly used several times unnecessarily in elastic_master.py and elastic_slave.py
- The comments and descriptions in elastic-site.xml have multiple suggested improvements.
- The comment description in common-services/METRON/0.3.0/configuration/metron-env.xml for "es_hosts" should clarify that this should be a list of ES Master hosts, not any ES hosts.
Attachments
Issue Links
- is cloned by
-
METRON-865 Additional Mpack bug fixes and improvements, that affect Ambari database
-
- Done
-
- links to
