METRON-793 migrated the storm topologies to the storm-kafka-client spout, which supports kerberos. To complete the kerberos work on the existing topologies, we need to be able to enable the spouts and kafka writers to use security protocols other than PLAINTEXT. Also, enabling auto-renew plugins for storm will enable the topologies to run for extended durations in a kerberized cluster.
This work was inspired by a portion of the investigatory work done at https://github.com/dlyle65535/incubator-metron/tree/kerb-testing?files=1 by:
This carves out a specific piece of that functionality with the following differences:
- The mpack work is not included, but the properties are set up to enable it as a follow-on
- It presumes
METRON-793, so it uses storm-kafka-client rather than storm-kafka
- It adds a flag when starting the parsers to pass the security protocol and sets up the writers and the spout automatically rather than relying on the set of extra kafka configs (though both approaches would work here).