[DRILL-8149] format-excel plugin needs to support POI IOUtils byte array overrides to support big files

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.19.0
• Fix Version/s: 1.21.0
• Component/s: Execution - Data Types
• Assignee: Unassigned
• Reporter: PJ Fanning
• Created: 24/Feb/22 13:28
• Updated: 12/Jul/22 06:59
• Resolved: 12/Jul/22 06:59

Description

https://poi.apache.org/components/configuration.html - see org.apache.poi.util.IOUtils.setByteArrayMaxOverride(int maxOverride)

Core POI code tries to set limits on resource allocations. excel-streaming-reader may not be as heavily affected by these settings because it only used parts of the core POI codebase.

POI 5.2.1 (due in next few weeks) fixes a few issues but there is some evidence that core POI users are hitting issues when loading large files and having to set  the byte array max override setting.

I can do some testing of the format-excel plugin to see if it can hit these issues with large files.