Description
This use-case is part of KIP-11 (UC-1)
This should allow topology based federation from one knox instance to another. This would be particularly useful for federating on-prem instances with cloud based ones.
Excerpt from UC-1
The key improvement needed for this usecase is the ability to override the dispatch mechanism for each service configured within a topology such that a single normalized dispatch is used across all services exposed by the topology. This single dispatch would probably just need to implement an outgoing version of our Header based Preauth SSO Provider with Client Cert over TLS.
The cloud Knox instance/s will need a topology that is configured with the Header based Preauth Provider and have the public key of the on-prem instance in its truststore.
This will enable things like:
- WebHDFS calls to an on-prem Knox actually redispatches the calls the cloud instance/s and results in files being put to or read from HDFS in the cloud.
- Spark jobs submitted to Livy through on-prem instances actually redispatch and are submitted as cloud workloads
- MapReduce jobs submitted to YARN RM through Knox will be submitted as workloads to the cloud
Attachments
Issue Links
- duplicates
-
KNOX-1339 Support for cloud federation
-
- Reopened
-