Bug 49940 - Sometimes writing twice to a .xlsx file throws org.apache.xmlbeans.impl.values.XmlValueDisconnectedException
Summary: Sometimes writing twice to a .xlsx file throws org.apache.xmlbeans.impl.value...
Status: RESOLVED FIXED
Alias: None
Product: POI
Classification: Unclassified
Component: XSSF (show other bugs)
Version: 3.8-FINAL
Hardware: PC All
: P2 major with 15 votes (vote)
Target Milestone: ---
Assignee: POI Developers List
URL:
Keywords:
: 46662 (view as bug list)
Depends on:
Blocks: 51158
  Show dependency tree
 
Reported: 2010-09-16 07:41 UTC by Vineet Potdar
Modified: 2016-01-07 07:44 UTC (History)
6 users (show)



Attachments
(patch) Code in org.apache.poi.xssf.usermodel.XSSFSheet (Library poi-ooxml-3.9) (240 bytes, text/plain)
2013-05-10 22:18 UTC, Jolteon
Details
Update to StylesTable to reuse existing container objects. (2.65 KB, patch)
2013-09-26 12:24 UTC, Dean Walker
Details | Diff
Patch proposal for solving this problem after cloneSheet() call (821 bytes, patch)
2014-05-06 10:37 UTC, Alessandro Guarascio
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Vineet Potdar 2010-09-16 07:41:12 UTC
I am trying to read and write to a .xlsx file from two separate Java methods. The first write goes through fine, at the second write (workbook.write(fileoutputstream) throws the error:
org.apache.xmlbeans.impl.values.XmlValueDisconnectedException

Stacktrace:

Exception in thread "main" org.apache.xmlbeans.impl.values.XmlValueDisconnectedException
	at org.apache.xmlbeans.impl.values.XmlObjectBase.check_orphaned(XmlObjectBase.java:1213)
	at org.apache.xmlbeans.impl.values.XmlObjectBase.newCursor(XmlObjectBase.java:243)
	at org.apache.xmlbeans.impl.values.XmlComplexContentImpl.arraySetterHelper(XmlComplexContentImpl.java:1073)
	at org.openxmlformats.schemas.spreadsheetml.x2006.main.impl.CTFontsImpl.setFontArray(Unknown Source)
	at org.apache.poi.xssf.model.StylesTable.writeTo(StylesTable.java:319)
	at org.apache.poi.xssf.model.StylesTable.commit(StylesTable.java:377)
	at org.apache.poi.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:177)
	at org.apache.poi.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:181)
	at org.apache.poi.POIXMLDocument.write(POIXMLDocument.java:214)
	at attendancerecorder.code.StudentRosterTest.checkOutStudent(StudentRosterTest.java:230)
	at attendancerecorder.code.AttendanceRecorderView.main(AttendanceRecorderView.java:46)
Comment 1 Vineet Potdar 2010-09-16 07:58:23 UTC
Forgot to add that it did work the first few times. Then suddenly it started throwing the error
Comment 2 Nick Burch 2010-09-16 09:48:44 UTC
I've confirmed the bug, and added a failing, disabled unit test to xssf/usermodel/TestXSSFBugs

The problem is coming from deep within xmlbeans, so will leave it for someone who knows more about that to look into
Comment 3 Chandan 2011-11-25 11:24:25 UTC
Can I have any updates on it? I am also facing the same issue. writing a fileOutputStream into a workbook works fine for the first time, but throws this exception in second attempt. This issue is critical. Would like to know the possible causes for this exception. A quick help is highly appreciated.
Comment 4 Nick Burch 2011-11-25 11:56:50 UTC
I think we're still not sure if this is an XMLBeans bug (either runtime or schema), or a problem with how we're calling XMLBeans

If you do make progress with identifying the cause of this, or the solution, please do report back!
Comment 5 Vivek 2012-04-04 13:27:42 UTC
Hi,

There is a need for our applications to be comaptible with Excel 2007 along
with 2003. 

We have a multiple write functionality and with the POI 3.7 using Workbook and
WorkbookFactory class works fine for 2003 version but fails for 2007 version.
We get an XMLValueDisconnectedException when we try to write the same workbook
twice.


Looks like there is a xmlbeans bug or may be POI calling xmlbeans...
Is there any code of POI calling xmlbeans that triggers the problem...

Any help in this view is appreciated.

Revert in case you need any further info.

Regards
Vivek Majithia.
Comment 6 Matt Lancaster 2013-01-03 16:15:30 UTC
This bug has been around for quite a while.  I'm still seeing this issue as of 1/3/2013 POI-3.8.  Any idea as to when this will be updated?  I'm not able to do any writing to an XMLFile with the workbook.write functionality.

Thanks,
Matt
Comment 7 Nick Burch 2013-01-07 09:56:44 UTC
This bug will remain open until someone with the time is able to work out first if it's a bug in XMLBeans, or a bug in how POI calls XMLBeans, then secondly work to get the issue (once tracked down) resolved. If this bug affects you, then you're a prime candidate to help!
Comment 8 Dr Meier 2013-03-08 07:43:26 UTC
I had the same problem too.

Exception in thread "main" org.apache.xmlbeans.impl.values.XmlValueDisconnectedException
	at org.apache.xmlbeans.impl.values.XmlObjectBase.check_orphaned(XmlObjectBase.java:1258)

XmlBeans Version 2.6.0.

I found out that i had the problem with an element that was there over 1000 times in a list.

As soon as i reduced the amount to 523 it works. As soon as i increased it over 524 it stops.

The filesize itself doesn't seem to be the problem ~ 500kb.
Comment 9 Dr Meier 2013-03-14 08:10:28 UTC
Now i had the problem in another context. I used the "getXXXXArray()" method to iterate over a list of XML Elements.

I removed an element with the removeXXXX() method.

As soon as i forgot to refresh the Array afterwards with .getXXXXArray() the XmlValueDisconnectedException happens again.

After the correction the code works fine:

DTCVALUE[] dtcValues = envData.getDTCVALUES().getDTCVALUEArray();
for(int k=dtcValues.length-1; k > 0 ;k--) {
	DTCVALUE dtcValue = dtcValues[k];
	if(!dtcMap.containsKey(dtcValue.getStringValue())){
		dtcMap.put(dtcValue.getStringValue(), envData);
	}else{
		DTCVALUES vals = envData.getDTCVALUES();
		vals.removeDTCVALUE(k);

		// !!!!!!! This is necessary, otherwise the Disconnected
                // Exception occors. !!!!!!!
		dtcValues = envData.getDTCVALUES().getDTCVALUEArray();
	}
}
Comment 10 Dr Meier 2013-03-14 08:20:08 UTC
(In reply to comment #8)
> I had the same problem too.
> 
> Exception in thread "main"
> org.apache.xmlbeans.impl.values.XmlValueDisconnectedException
> 	at
> org.apache.xmlbeans.impl.values.XmlObjectBase.check_orphaned(XmlObjectBase.
> java:1258)
> 
> XmlBeans Version 2.6.0.
> 
> I found out that i had the problem with an element that was there over 1000
> times in a list.
> 
> As soon as i reduced the amount to 523 it works. As soon as i increased it
> over 524 it stops.
> 
> The filesize itself doesn't seem to be the problem ~ 500kb.

This also could be solved with my statement in Comment #9. It doesn't have something to do with the size, simply the reassigment of the array is necesarry.
Comment 11 Dr Meier 2013-03-14 13:13:59 UTC
(In reply to comment #9)
> Now i had the problem in another context. I used the "getXXXXArray()" method
> to iterate over a list of XML Elements.
> 
> I removed an element with the removeXXXX() method.
> 
> As soon as i forgot to refresh the Array afterwards with .getXXXXArray() the
> XmlValueDisconnectedException happens again.
> 
> After the correction the code works fine:
> 
> DTCVALUE[] dtcValues = envData.getDTCVALUES().getDTCVALUEArray();
> for(int k=dtcValues.length-1; k > 0 ;k--) {
> 	DTCVALUE dtcValue = dtcValues[k];
> 	if(!dtcMap.containsKey(dtcValue.getStringValue())){
> 		dtcMap.put(dtcValue.getStringValue(), envData);
> 	}else{
> 		DTCVALUES vals = envData.getDTCVALUES();
> 		vals.removeDTCVALUE(k);
> 
> 		// !!!!!!! This is necessary, otherwise the Disconnected
>                 // Exception occors. !!!!!!!
> 		dtcValues = envData.getDTCVALUES().getDTCVALUEArray();
> 	}
> }


Loop must run until "k >= 0", sorry!
Comment 12 Jolteon 2013-05-09 21:52:40 UTC
Not sure about my analysis, but I got that error when I tried to duplicate a Row. I did createRow(rowIndex+1). By that, I meant "insert a Row at index rowIndex+1". After that I had to clone the value of each cell from the Row at rowIndex to the Row at rowIndex+1. However I already had a Row at rowIndex+1 (before the createRow), and I think that the previous cells on this Row get lost and don't point on anything anymore.

If I am correct, then it would not be a problem in the XMLBeans library. How does the Apache Poi library handle the creation of a Row (or a Cell) if this Row/Cell already exists ? Does it overwrite it ? If yes, the previous Row/Cell object becomes indeed obsolete (which is freaking problematic, since each createRow/createCell can make great damage)

Again, that's just a guess, and I have to dig a little further in my code ...
Comment 13 Jolteon 2013-05-10 21:45:25 UTC
(In reply to comment #12)

Now a minimal code that triggers the error :

Row row = _sheet.getRow(0);     // defined
Cell orphaned = row.getCell(0); // defined
Row row1 = _sheet.createRow(0); // we lose orphaned
System.out.println(orphaned);   // we trigger the exception
Comment 14 Jolteon 2013-05-10 22:18:10 UTC
Created attachment 30271 [details]
(patch) Code in org.apache.poi.xssf.usermodel.XSSFSheet (Library poi-ooxml-3.9)

I don't know exactly what the CTRow.Factory.newInstance() does, but I am pretty sure that it creates a new CTRow which is decorellated from the first one.

If we look at all the cells in the Row and remove them, it should do the trick !
Comment 15 Nick Burch 2013-06-25 15:56:22 UTC
(In reply to Jolteon from comment #13)
> (In reply to comment #12)
> 
> Now a minimal code that triggers the error :
> 
> Row row = _sheet.getRow(0);     // defined
> Cell orphaned = row.getCell(0); // defined
> Row row1 = _sheet.createRow(0); // we lose orphaned
> System.out.println(orphaned);   // we trigger the exception

You shouldn't be calling createRow for a row that already exists. I'm surprised that it doesn't blow up earlier...

However, there does seem to be some logic in XSSFSheet for this case, which I don't understand.

Does anyone know the use case for calling createRow with the index of a row that already exists?
Comment 16 liny 2013-07-09 01:32:41 UTC
It also occurred when calling cloneSheet() method before write():
--------------------------------------------------------------------------
      wb.cloneSheet(0);
      File destPath = new File(outputFilePathName).getParentFile();
      if (!destPath.exists())
      {
        log.info("Created path {}? {}", destPath.getAbsolutePath(), destPath.mkdirs());
      }
      wb.write(new FileOutputStream(outputFilePathName));
--------------------------------------------------------------------------
Comment 17 Dominik Stadler 2013-08-05 20:18:57 UTC
*** Bug 46662 has been marked as a duplicate of this bug. ***
Comment 18 Dominik Stadler 2013-09-09 13:57:58 UTC
There are multiple things that can lead to these XmlValueDisconnectedException, they basically indicate that an XmlBeans-object was removed from the tree of object, but later still used, either because it is still referenced or was incorrectly removed. 

A few such cases were fixed over the last weeks, however this one still remains. 

It seems to be located in StylesTable.writeTo() where all the styles are copied around, not sure why this is necessary in the first place...
Comment 19 Dean Walker 2013-09-26 12:19:34 UTC
I've seen this behaviour in various places (I'm using POI 3.10 Beta 1).
This disconnection happens during the first save where new 'container' objects are created.
For example:
		// Fonts
		CTFonts ctFonts = CTFonts.Factory.newInstance();
		ctFonts.setCount(fonts.size());
		CTFont[] ctfnt = new CTFont[fonts.size()];
		idx = 0;
		for(XSSFFont f : fonts) ctfnt[idx++] = f.getCTFont();
		ctFonts.setFontArray(ctfnt);
		styleSheet.setFonts(ctFonts);

Here CTFonts is a container for a CTFont array.
If we reuse the existing container, the problem goes away, example:

		// Fonts
		CTFonts ctFonts = styleSheet.getFonts();
		if (ctFonts == null) {
			ctFonts = CTFonts.Factory.newInstance();
		}
		ctFonts.setCount(fonts.size());
		CTFont[] ctfnt = new CTFont[fonts.size()];
		idx = 0;
		for(XSSFFont f : fonts) ctfnt[idx++] = f.getCTFont();
		ctFonts.setFontArray(ctfnt);
		styleSheet.setFonts(ctFonts);

This works for my use case where I read an existing spreadsheet, update cell values only, and then save the modified spreadsheet. I've not tried the use case where fonts (or other objects) are programatically modified or added or deleted.
There are numerous places in StylesTable.writeTo() where I have applied this same solution.
I also encountered the same problem with named ranges and applied the same solution to XSSFWorkBook.saveNamedRanges().
There may be other places where the same symptom appears depending on the features you use in your spreadsheet.
This solution is not ideal (it cures the symptom rather than the cause) but I hope this information will help resolve this bug.
Comment 20 Dean Walker 2013-09-26 12:24:26 UTC
Created attachment 30885 [details]
Update to StylesTable to reuse existing container objects.

Suspect similar changes need to be applied to 'Formats' if your spreadsheet has custom formats
Comment 21 Dominik Stadler 2013-10-26 07:34:06 UTC
I have applied this patch now as r1535938. 

As for the other places where this potentially can happen, I looked at a few of them and could not immediately decide if the problem can happen for them as well or not. On the other hand we are working on some "stress-tests" which will run the set of sample-files that we use for testing and do various things on them, this way we should uncover more of these cases if there are still places left where it can happen. 

Additional patches with test-cases are very welcome as well, naturally!
Comment 22 Alessandro Guarascio 2014-05-06 10:37:54 UTC
Created attachment 31596 [details]
Patch proposal for solving this problem after cloneSheet() call

Here is a patch proposal for POI 3.10 to solve the problem occurring after cloneSheet() call.
Comment 23 Avik Dey 2016-01-07 06:48:18 UTC
I have created an object of the Xls_Reader only once, and saved it at the last, still I got the exception...

I am creating a .xlsx file every time I am trying to write with date and time stamp.

Still I have encountered the error.

I am not using the same file to write twice into...

ERROR
=======

org.apache.xmlbeans.impl.values.XmlValueDisconnectedException
	at org.apache.xmlbeans.impl.values.XmlObjectBase.check_orphaned(XmlObjectBase.java:1244)
	at org.apache.xmlbeans.impl.values.XmlObjectBase.newCursor(XmlObjectBase.java:272)
	at org.apache.xmlbeans.impl.values.XmlComplexContentImpl.arraySetterHelper(XmlComplexContentImpl.java:1073)
	at org.openxmlformats.schemas.spreadsheetml.x2006.main.impl.CTFontsImpl.setFontArray(Unknown Source)
	at org.apache.poi.xssf.model.StylesTable.writeTo(StylesTable.java:319)
	at org.apache.poi.xssf.model.StylesTable.commit(StylesTable.java:377)
	at org.apache.poi.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:177)
	at org.apache.poi.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:181)
	at org.apache.poi.POIXMLDocument.write(POIXMLDocument.java:214)
	at com.soapuitutorial.util.Xls_Reader.save(Xls_Reader.java:547)
	at com.soapuitutorial.script.ProjectTestRunner.main(ProjectTestRunner.java:73)


CODE - save()
==============

public void save(){
		try{
		fileOut = new FileOutputStream(path);

		workbook.write(fileOut);
		//fileOut.flush();
	    fileOut.close();
	    fileOut=null;
	    }catch(Exception e){
	    	e.printStackTrace();
	    }
	}
Comment 24 Dominik Stadler 2016-01-07 07:44:58 UTC
Please don't post onto an old bug that is set to resolved, it likely will go unnoticed. If you still have an issue when using the latest released version of POI, please post full steps to reproduce the problem (best as self-contained unit test) together with sample files if needed in a new bug report.