Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
There are a handful of things that happen in NiFi (and stateless) startup that take quite a while that can be improved:
- We unpack all NAR's if they are not already unpacked. If they are unpacked, we perform a hash of the nar to ensure that it matches with what was unpacked. With stateless, though, we download nars on demand and name them a temporary file until the download completes, then rename them. As a result, we don't expect any hash conflicts unless it's a SNAPSHOT nar. We should avoid calculating hashes against all of the nars on startup.
- We use the ServiceLoader to load all extensions. Given the number of extensions that we have and the prevalence of things like ObjectMapper as member variables, this can be rather expensive. We should instead just read the ServiceLoader file and create the objects on-demand. The temp components are needed for generating documentation, for example. But otherwise they are not needed. And stateless doesn't bother generating docs (since there's no UI to view them). This avoids creating lots of expensive objects and can also prevent loading huge numbers of classes into memory.
- StringEncryptor is almost never used in stateless. We should lazy load it if needed.
- Controller Service enabling is very slow in stateless. Currently, it asynchronously starts all services. Then it keeps polling to see if services are enabled and if not sleeps for a short period. We should instead move to a method of using Futures or wait/notify, etc. so that service enablement can completely quickly.
