Bug 46280 - Unexpected ContinueRecord in RowRecordsAggregate when PivotTables are present
Summary: Unexpected ContinueRecord in RowRecordsAggregate when PivotTables are present
Status: RESOLVED FIXED
Alias: None
Product: POI
Classification: Unclassified
Component: HSSF (show other bugs)
Version: 3.5-dev
Hardware: PC Windows XP
: P2 normal (vote)
Target Milestone: ---
Assignee: POI Developers List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-11-24 14:12 UTC by Josh Micich
Modified: 2008-11-25 11:00 UTC (History)
0 users



Attachments
BiffViewer analysis (90.71 KB, application/x-gzip)
2008-11-25 03:17 UTC, Axel Rose
Details
summary of previous attachment (4.03 KB, text/plain)
2008-11-25 11:00 UTC, Josh Micich
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Josh Micich 2008-11-24 14:12:21 UTC
This bugzilla was opened in response to the mailing list posting made by Axel Roeslein:

http://mail-archives.apache.org/mod_mbox/poi-user/200811.mbox/%3C29bf25ca0811200724y620db645p1783ac1c0bd5085d@mail.gmail.com%3E

The following stack trace occurred:

Exception in thread "main" java.lang.RuntimeException: Unexpected record type (org.apache.poi.hssf.record.ContinueRecord)
       at org.apache.poi.hssf.record.aggregates.RowRecordsAggregate.<init>(RowRecordsAggregate.java:87)
       at org.apache.poi.hssf.model.Sheet.<init>(Sheet.java:197)
       at org.apache.poi.hssf.model.Sheet.createSheet(Sheet.java:159)
       at org.apache.poi.hssf.usermodel.HSSFWorkbook.<init>(HSSFWorkbook.java:286)
       at org.apache.poi.hssf.usermodel.HSSFWorkbook.<init>(HSSFWorkbook.java:200)
       at org.apache.poi.hssf.usermodel.HSSFWorkbook.<init>(HSSFWorkbook.java:182)

Unfortunately the workbook had sensitive data in it, so it couldn't be uploaded.  Re-saving in Excel makes the problem go away, and the problem has not been reproduced with another workbook yet.

Further investigation revealed that the offending ContinueRecord was an extension to a "SXLI - Line Item Array" (0x00B5) record, which is a PivotTable record.  Indeed, the workbook does contain a large pivot table.
Comment 1 Josh Micich 2008-11-24 15:05:42 UTC
Fix applied in svn r720318

The logic inside RowRecordsAggregate was changed to allow for ContinueRecords after any UnknownRecord.

The arrangement of PivotTable records in the worksheet stream is not discussed in the OOO documentation.  Brief investigation suggests that they come after all of the row-block records of the current sheet (regardless of the actually positioning of the pivot table(s) on the sheet).

However, Excel 2007 seems to put MSODRAWING(0x00EC) before the PivotTable records, and this should have prevented this bug, since RowBlocksReader stops at DrawingRecord.sid(0x00EC).  I am assuming that the offending file does not have an MSODRAWING record before the PivotTable records, so I added a fix in RecordOrderer.isEndOfRowBlock(int) to make sure PivotTable records are not included in the 'row blocks'.

My original assumption was that re-saving with Excel removed the offending ContinueRecord.  Now I am guessing that re-saving with Excel put the MSODRAWING record in the right place.

Comment 2 Axel Rose 2008-11-25 03:17:37 UTC
Created attachment 22935 [details]
BiffViewer analysis

I updated my sources from svn and rechecked. Reading works as expcected.

    I would like to know if MSODRAWING (0x00EC) is
    missing before the PivotTable records (0x00B* etc).

No, as far as I can see.

    Is the first PivotTable record 0x00B0?

No.

    Does re-saving with Excel insert MSODRAWING
    in the expected place?

What is the expected place?
 

Please see the attached file. It was produces outputting the BiffViewer result
to "note23.1" and then doing "grep sid=0x note23.1 > note23.sid.txt"
Comment 3 Josh Micich 2008-11-25 11:00:45 UTC
Created attachment 22947 [details]
summary of previous attachment

I have summarized the attached 'note23.sid.txt' to just the following records:
 - All BOFs and EOFs to see where the sheets start and stop
 - For each sheet containing SXVIEW, all Records from DBCELL to SXVIEW
 - All continued records (only 2 in the whole workbook)

In the summary, it can be seen that only in the fifth sheet is SXVIEW not preceeded with MSODRAWING. This also happens to be the only sheet with a continued PivotTable record (recno=7750). Both conditions had to occur together to make this bug occur.