[AMBARI-14049] Installing cluster with accumulo using blueprints causes accumulo tserver startup failure

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 2.1.2
• Component/s: blueprints

Environment

CentOS 6

• Assignee: Unassigned
• Reporter: Jasbir Dhillon
• Created: 24/Nov/15 19:34
• Updated: 01/Jun/16 20:15

Description

Installing accumulo on a cluster using blueprints causes accumulo tserver startup failure.

We have three host_groups.

Host_group_1 consists of the components: PIG, KAFKA_BROKER, HISTORYSERVER, OOZIE_CLIENT, NAMENODE, HCAT, METRICS_MONITOR, AMBARI_SERVER, ACCUMULO_MASTER, APP_TIMELINE_SERVER, ACCUMULO_MONITOR, HIVE_CLIENT, HDFS_CLIENT, METRICS_COLLECTOR, FLUME_HANDLER, DATANODE, WEBHCAT_SERVER, ZOOKEEPER_SERVER, ZOOKEEPER_CLIENT, HIVE_SERVER, OOZIE_SERVER, TEZ_CLIENT, HIVE_METASTORE, ACCUMULO_TRACER, YARN_CLIENT, ACCUMULO_GC, MAPREDUCE2_CLIENT, MYSQL_SERVER, ACCUMULO_CLIENT

Host_group_2 consists of the components: ZOOKEEPER_SERVER, NODEMANAGER, METRICS_MONITOR, SECONDARY_NAMENODE, DATANODE, ACCUMULO_TSERVER.

Host_group_3 consists of the components: ZOOKEEPER_SERVER, NODEMANAGER, METRICS_MONITOR, DATANODE, ACCUMULO_TSERVER

The accumulo tservers fail due to the following error. The accumulo tserver start up task should wait for the namenode to start up which is running on host group 1. I tested the above configuration without blueprints and it works fine as host group 2 and host group 3 wait for tasks on host group 1 to complete prior to starting up the components.

2015-11-24 18:59:05,807 [zookeeper.ZooUtil] ERROR: Problem reading instance id out of hdfs at hdfs://<host group 1 server name>:8020/apps/accumulo/data/instance_id
java.net.ConnectException: Call From <host group 1 server name>/<ip address> to <host group 1 server name>:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)