Details
-
New Feature
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
New
Description
In some situations we would like to store the whole text, but the whole text won't always fit in memory so we can't create a String. Likewise for storing binary, it would sometimes be better if we didn't have to read into a byte[] up-front (even when it doesn't use much memory, it increases the number of copies made and adds burden to GC.)
FieldsWriter currently writes the length at the start of the chunks though, so I don't know whether it would be possible to seek back and write the length after writing the data.
It would also be useful to use this in conjunction with compression, both for Reader and InputStream types. And when retrieving the field, it should be possible to create a Reader without reading the entire String into memory up-front.