[EAGLE-469] make dataSource configuration extremely simple

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: v0.5.0
• Fix Version/s: v0.5.1
• Assignee: Edward Zhang
• Reporter: Edward Zhang
• Due: 19/Aug/16
• Created: 16/Aug/16 00:50
• Updated: 20/Jul/17 17:52

Description

Today dataSource configuration is very cumbersome, we actually can reduce significantly the size of config entries as most of configurations use the default values.
"dataSourceConfig":

{ "topic" : "sandbox_hbase_security_log", "zkConnection" : "sandbox.hortonworks.com:2181", "zkConnectionTimeoutMS" : 15000, "consumerGroupId" : "EagleConsumer", "fetchSize" : 1048586, "deserializerClass" : "org.apache.eagle.security.hbase.HbaseAuditLogKafkaDeserializer", "transactionZKServers" : "sandbox.hortonworks.com", "transactionZKPort" : 2181, "transactionZKRoot" : "/consumers", "consumerGroupId" : "eagle.hbasesecurity.consumer", "transactionStateUpdateMS" : 2000 }

The new data source config would be like:
"dataSourceConfig":

{ "topic" : "hbase_audit_log", "zkConnection" : "server.eagle.apache.org:2181", "txZkServers" : "server.eagle.apache.org:2181", "schemeCls" : "storm.kafka.StringScheme" }

Issue Links

• links to: GitHub Pull Request #353