Description
While verifying DAFFODIL-2455 - - Large CSV file causes "Attempting to backtrack too far" exception, found that unparsing the successfully parsed 800mb CSV files infoset ran out of memory.
Increased the DAFFODIL_JAVA_OPTS memory setting several time up to 32gb and tried unparsing the infoset, each time running out of memory. Ran on test platform which has 90+GB of memory.
Parsed and unparsed using the shema from dfdl-shemas/dfdl-csv repo.
The 800gb csv file (csv_data800m.csv) gzipped.