Documentation for Kafka Channel

+1

Minor one, if we can put the URI format for zookeeper as well

*zookeeperConnect* – URI of ZooKeeper used by Kafka cluster. The format is comma separated list of hostname:port

Thanks for writing these docs!

A few minor comments:

• The example shows kafka-1:9092 etc. That should be a full hostname, correct? (for both ZK and broker list).
• Mention that class that is used when parseAsFlumeEvent is true. This would be required if we just want Flume to write to Kafka and not remove it from the channel - so other consumers are able to remove the data (Log4j->Flume->Kafka etc). The class is org.apache.flume.source.avro.AvroFlumeEvent provided by the flume-ng-sdk artifact.
• Worth mentioning how multiple channels must use the same topic and group to ensure that when one agent fails another can remove the data.
• readSmallestOffset matters only if parseAsFlumeEvent is false. We should also mention why it is better to keep it to false (or not set it at all) if parseAsFlumeEvent is false.
• Mention that producer.type will always be sync and other properties which we override anyway - even if the customer has overriden it.
• Mention that by using Kafka Channel -> HDFS/HBase/Solr sink, Kafka+Flume provides low latency, fault-tolerant writes to the corresponding system.

Thanks for the detailed review!

Fixed most issues raised by Ashish and Hari.

Exceptions:
1) any hostname that resolves is valid for zookeeper and broker. I didn't include FQD for brevity and clarity
2) I'd rather not document properties that we don't want users to override.
Lets view it as internal implementation detail? I feel that documenting invites fiddling, and I'd rather they won't fiddle here.

+1. There were a couple of formatting errors. I fixed those and am committing it.

Commit fdd61e840b004ca1b5c23b28e8d9eeda300f56f7 in flume's branch refs/heads/trunk from Hari Shreedharan
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=fdd61e8 ]

FLUME-2523. Document Kafka channel

(Gwen Shapira via Hari)

Commit ae2ded9ce6e112f8a423853e6781b1e02862ab3c in flume's branch refs/heads/flume-1.6 from Hari Shreedharan
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=ae2ded9 ]

FLUME-2523. Document Kafka channel

(Gwen Shapira via Hari)

Committed! Thanks Gwen!

UNSTABLE: Integrated in Flume-trunk-hbase-98 #43 (See https://builds.apache.org/job/Flume-trunk-hbase-98/43/)
FLUME-2523. Document Kafka channel (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=fdd61e840b004ca1b5c23b28e8d9eeda300f56f7)

• flume-ng-doc/sphinx/FlumeUserGuide.rst

SUCCESS: Integrated in flume-trunk #686 (See https://builds.apache.org/job/flume-trunk/686/)
FLUME-2523. Document Kafka channel (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=fdd61e840b004ca1b5c23b28e8d9eeda300f56f7)

• flume-ng-doc/sphinx/FlumeUserGuide.rst