[NIFI-7001] Guard against loading/operating on/serializing large files in EC toolkit

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.13.0
• Component/s: Tools and Build
• Labels: documentation security toolkit validation
• Assignee: Unassigned
• Reporter: Andy LoPresto
• Created: 09/Jan/20 17:57
• Updated: 28/Sep/21 16:19
• Resolved: 28/Sep/21 16:19

Description

The EC toolkit did not anticipate the possibility of a flow.xml.gz on the order of magnitude of 700 MB. Many serialization/deserialization and string manipulation operations occur assuming that the content is of manageable size. Users have demonstrated that this is not a safe assumption.

We should introduce the following protections:

1. log debug/info statements at file load, encrypt replacement and file write for the size of the content for visibility
2. simple conditional checks at file load to ensure the heap is properly sized and the file content size is reasonable
3. if the size is too large, print a helpful message and direct users to the Toolkit Guide for further information

Issue Links

• is fixed by: NIFI-6999 Encrypt Config Toolkit fails on very large flow.xml.gz files