Description
Currently, Atlas hooks that run in Hive, Falconm Storm, Sqoop processes reference atlas-application.properties file from Atlas server config location - /etc/atlas/conf/atlas-application.properties.
Not all properties in /etc/atlas/conf/atlas-application.properties are required in hooks and some of these properties are sensitive enough not to expose them to hooks/clients.
To address this concern:
1. atlas-application.properties should be added as a config section in each of the host component's that run Atlas hook - Hive, Storm, Falcon, Sqoop
2. These new config sections will only include properties that are required to the respective hooks
3. During initial deployment, Ambari will initialize these properties with values in Atlas server configuration.
For each one of those services, create a config type called ${service}-atlas-application.properties that will be saved to /etc/${service}/conf/application.properties
These are the default values,
Falcon
atlas.hook.falcon.synchronous=false
atlas.hook.falcon.numRetries=3
atlas.hook.falcon.minThreads=5
atlas.hook.falcon.maxThreads=5
atlas.hook.falcon.keepAliveTime=10
atlas.hook.falcon.queueSize
Storm
atlas.hook.storm.numRetries=3
Hive
atlas.hook.hive.synchronous=false
atlas.hook.hive.numRetries=3
atlas.hook.hive.minThreads=5
atlas.hook.hive.maxThreads=5
atlas.hook.hive.keepAliveTime=10
atlas.hook.hive.queueSize=10000
Common for all hooks
atlas.kafka.zookeeper.connect=
atlas.kafka.bootstrap.servers=
atlas.kafka.zookeeper.session.timeout.ms=400
atlas.kafka.zookeeper.connection.timeout.ms=200
atlas.kafka.zookeeper.sync.time.ms=20
atlas.kafka.hook.group.id=atlas
atlas.notification.create.topics=true
atlas.notification.replicas=1
atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES
atlas.notification.kafka.service.principal=kafka/_HOST@EXAMPLE.COM
atlas.notification.kafka.keytab.location=/etc/security/keytabs/kafka.service.keytab
atlas.jaas.KafkaClient.loginModuleName = com.sun.security.auth.module.Krb5LoginModule
atlas.jaas.KafkaClient.loginModuleControlFlag = required
atlas.jaas.KafkaClient.option.useKeyTab = true
atlas.jaas.KafkaClient.option.storeKey = true
atlas.jaas.KafkaClient.option.serviceName = kafka
atlas.jaas.KafkaClient.option.keyTab = /etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_HOST@EXAMPLE.COM
atlas.cluster.name=
Attachments
Attachments
Issue Links
- links to