Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
The instructions on how to configure the JMS support were not found sufficient during Oozie integration.
Things that need to be covered:
Server configuration to enable JMS notifications
In addition to the changes to hive-site.xml file mentioned in #3 you will have to make (add/modify) the following changes to the hive-site.xml file of your HCatalog server to turn on notifications
<property>
<name>hive.metastore.event.expiry.duration</name>
<value>300L</value>
<description>Duration after which events expire from events table (in seconds)</description>
</property>
<property>
<name>hive.metastore.event.clean.freq</name>
<value>360L</value>
<description>Frequency at which timer task runs to purge expired events in metastore(in seconds).</description>
</property>
<property>
<name>msgbus.brokerurl</name>
<value>tcp://localhost:61616</value>
<description></description>
</property>
<property>
<name>msgbus.username</name>
<value></value>
<description></description>
</property>
<property>
<name>msgbus.password</name>
<value></value>
<description></description>
</property>
For the server to start with support for notifications, following must be in the classpath:
activemq jar and jndi.properties file with properties suitably configured for notifications.
1. HCatalog server start script is $YOUR_HCAT_SERVER/share/hcatalog/scripts/hcat_server_start.sh
2. This script expects classpath to be set by the AUX_CLASSPATH environment variable.
3. Therefore set AUX_CLASSPATH to satisfy #1.1 and #1.2 above.
4. jndi.properties file is located at $YOUR_HCAT_SERVER/etc/hcatalog/jndi.properties
5. You need to uncomment and set the following properties in this file: -
java.naming.factory.initial = org.apache.activemq.jndi.ActiveMQInitialContextFactory
java.naming.provider.url = tcp://localhost:61616 <this is activemq url in your setup>
Topic names
If tables are created while the server is configured for notifications, a default topic name is automatically set as table property. To use notifications with tables created previously (previous HCatalog installations or created prior to enabling notifications), you will have to manually set a topic name, an example will be: -
$YOUR_HCAT_CLIENT_HOME/bin/hcat -e "ALTER TABLE access SET hcat.msgbus.topic.name=$TOPIC_NAME"
and then you need to configure your activemq Consumer(s) to listen for
messages on the topic you gave in $TOPIC_NAME. A good default policy
for TOPIC_NAME = "$database.$table" (that is a literal dot)