Uploaded image for project: 'Metron'
  1. Metron
  2. METRON-797

Pass security.protocol and enable auto-renew for the storm topologies

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Done
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Labels:

      Description

      METRON-793 migrated the storm topologies to the storm-kafka-client spout, which supports kerberos. To complete the kerberos work on the existing topologies, we need to be able to enable the spouts and kafka writers to use security protocols other than PLAINTEXT. Also, enabling auto-renew plugins for storm will enable the topologies to run for extended durations in a kerberized cluster.

      This work was inspired by a portion of the investigatory work done at https://github.com/dlyle65535/incubator-metron/tree/kerb-testing?files=1 by:

      • @dlyle65535
      • @merrimanr
      • @mmiklavc

      This carves out a specific piece of that functionality with the following differences:

      • The mpack work is not included, but the properties are set up to enable it as a follow-on
      • It presumes METRON-793, so it uses storm-kafka-client rather than storm-kafka
      • It adds a flag when starting the parsers to pass the security protocol and sets up the writers and the spout automatically rather than relying on the set of extra kafka configs (though both approaches would work here).

      NOTE: This does not encompass MPack changes to enable kerberos (METRON-799) or fix the sensors to work with a kerberized kafka (METRON-798). That would be follow-on work.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                cestella Casey Stella
                Reporter:
                cestella Casey Stella
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: