Details

      Description

      I'm modifying file metadata .ods, the application works correctly for files under 3 MB but in the case of files larger of 2,8 MB it creates a file 2 KB and get locked by the java process. No java error display.
      My code is as follows:

      doc = OdfDocument.loadDocument(new File("C:\tmp\ficheroIn.ods"));
      OdfFileDom metadom = doc.getMetaDom();
      Meta metadata = new Meta(metadom);

      metadata.setTitle(plantilla.getTitulo());
      metadata.setSubject(plantilla.getAsunto());
      metadata.setCreator(plantilla.getModificadoPor());
      metadata.setDescription(plantilla.getComentarios());

      doc.save("C:\tmp\ficheroOut.ods");

        Activity

        Hide
        Svante Schubert added a comment -

        In case the failure was again an out-of-memory exception it might happen anywhere (when the programms runs out-of-memory, likely dependent on machine and other applications running).
        The ZipHelper is equal to mine version, the OdfPackage differs a lot as RDF and Signature/Encrytion is not included in my branch, therefore a merge would need more time, which I do not possess atm. Perhaps early October, depends on the projects I gain money from, which have obviously higher priority, therefore sorry I can not say.

        Show
        Svante Schubert added a comment - In case the failure was again an out-of-memory exception it might happen anywhere (when the programms runs out-of-memory, likely dependent on machine and other applications running). The ZipHelper is equal to mine version, the OdfPackage differs a lot as RDF and Signature/Encrytion is not included in my branch, therefore a merge would need more time, which I do not possess atm. Perhaps early October, depends on the projects I gain money from, which have obviously higher priority, therefore sorry I can not say.
        Hide
        Jacinto Verdaguer added a comment -

        For files larger than 3 MB the method fails is:

        private InputStream readAsInputStream(ZipInputStream inputStream) throws IOException {
        ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
        if (outputStream != null) {
        byte[] buf = new byte[4096];
        int r = 0;
        while ((r = inputStream.read(buf, 0, 4096)) > -1)

        { outputStream.write(buf, 0, r); }

        inputStream.close();
        }
        return new ByteArrayInputStream(outputStream.toByteArray());
        }

        Specifically the line: outputStream.write(buf, 0, r);
        This method is in the class ZipHelper.java

        Show
        Jacinto Verdaguer added a comment - For files larger than 3 MB the method fails is: private InputStream readAsInputStream(ZipInputStream inputStream) throws IOException { ByteArrayOutputStream outputStream = new ByteArrayOutputStream(); if (outputStream != null) { byte[] buf = new byte [4096] ; int r = 0; while ((r = inputStream.read(buf, 0, 4096)) > -1) { outputStream.write(buf, 0, r); } inputStream.close(); } return new ByteArrayInputStream(outputStream.toByteArray()); } Specifically the line: outputStream.write(buf, 0, r); This method is in the class ZipHelper.java
        Hide
        Jacinto Verdaguer added a comment -

        Thank you very much for your help. When do you post the version with the correction?

        Show
        Jacinto Verdaguer added a comment - Thank you very much for your help. When do you post the version with the correction?
        Hide
        Svante Schubert added a comment -

        Seems it was by coincidence in the RDFa class as the memory went out, with the prior RDFa module the error was the same, but in a different place:
        java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.util.Arrays.copyOf(Arrays.java:2271)
        at java.util.zip.ZipCoder.getBytes(ZipCoder.java:89)
        at java.util.zip.ZipFile.getEntry(ZipFile.java:306)
        at java.util.jar.JarFile.getEntry(JarFile.java:226)
        at java.util.jar.JarFile.getJarEntry(JarFile.java:209)
        at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:832)
        at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:810)
        at sun.misc.URLClassPath.findResource(URLClassPath.java:176)
        at java.net.URLClassLoader$2.run(URLClassLoader.java:551)
        at java.net.URLClassLoader$2.run(URLClassLoader.java:549)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findResource(URLClassLoader.java:548)
        at java.lang.ClassLoader.getResource(ClassLoader.java:1139)
        at java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227)
        at javax.xml.stream.SecuritySupport$4.run(SecuritySupport.java:94)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.xml.stream.SecuritySupport.getResourceAsStream(SecuritySupport.java:87)
        at javax.xml.stream.FactoryFinder.findJarServiceProvider(FactoryFinder.java:317)
        at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:289)
        at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:213)
        at javax.xml.stream.XMLEventFactory.newInstance(XMLEventFactory.java:59)
        at net.rootdev.javardfa.Parser.<init>(Parser.java:43)
        at org.odftoolkit.odfdom.pkg.rdfa.RDFaParser.<init>(RDFaParser.java:63)
        at org.odftoolkit.odfdom.pkg.rdfa.DOMRDFaParser.<init>(DOMRDFaParser.java:44)
        at org.odftoolkit.odfdom.pkg.rdfa.DOMRDFaParser.createInstance(DOMRDFaParser.java:38)
        at org.odftoolkit.odfdom.pkg.OdfFileDom.updateInContentMetadataCache(OdfFileDom.java:691)
        at org.odftoolkit.odfdom.dom.element.text.TextParagraphElementBase.onInsertNode(TextParagraphElementBase.java:300)
        at org.odftoolkit.odfdom.pkg.OdfElement.onInsertNode(OdfElement.java:478)
        at org.odftoolkit.odfdom.pkg.OdfElement.insertBefore(OdfElement.java:490)
        at org.apache.xerces.dom.NodeImpl.appendChild(Unknown Source)
        at org.odftoolkit.odfdom.pkg.OdfFileSaxHandler.startElement(OdfFileSaxHandler.java:121)

        I will keep this issue in mind, when I move the changes back to the main repository.

        Show
        Svante Schubert added a comment - Seems it was by coincidence in the RDFa class as the memory went out, with the prior RDFa module the error was the same, but in a different place: java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.Arrays.copyOf(Arrays.java:2271) at java.util.zip.ZipCoder.getBytes(ZipCoder.java:89) at java.util.zip.ZipFile.getEntry(ZipFile.java:306) at java.util.jar.JarFile.getEntry(JarFile.java:226) at java.util.jar.JarFile.getJarEntry(JarFile.java:209) at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:832) at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:810) at sun.misc.URLClassPath.findResource(URLClassPath.java:176) at java.net.URLClassLoader$2.run(URLClassLoader.java:551) at java.net.URLClassLoader$2.run(URLClassLoader.java:549) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findResource(URLClassLoader.java:548) at java.lang.ClassLoader.getResource(ClassLoader.java:1139) at java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227) at javax.xml.stream.SecuritySupport$4.run(SecuritySupport.java:94) at java.security.AccessController.doPrivileged(Native Method) at javax.xml.stream.SecuritySupport.getResourceAsStream(SecuritySupport.java:87) at javax.xml.stream.FactoryFinder.findJarServiceProvider(FactoryFinder.java:317) at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:289) at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:213) at javax.xml.stream.XMLEventFactory.newInstance(XMLEventFactory.java:59) at net.rootdev.javardfa.Parser.<init>(Parser.java:43) at org.odftoolkit.odfdom.pkg.rdfa.RDFaParser.<init>(RDFaParser.java:63) at org.odftoolkit.odfdom.pkg.rdfa.DOMRDFaParser.<init>(DOMRDFaParser.java:44) at org.odftoolkit.odfdom.pkg.rdfa.DOMRDFaParser.createInstance(DOMRDFaParser.java:38) at org.odftoolkit.odfdom.pkg.OdfFileDom.updateInContentMetadataCache(OdfFileDom.java:691) at org.odftoolkit.odfdom.dom.element.text.TextParagraphElementBase.onInsertNode(TextParagraphElementBase.java:300) at org.odftoolkit.odfdom.pkg.OdfElement.onInsertNode(OdfElement.java:478) at org.odftoolkit.odfdom.pkg.OdfElement.insertBefore(OdfElement.java:490) at org.apache.xerces.dom.NodeImpl.appendChild(Unknown Source) at org.odftoolkit.odfdom.pkg.OdfFileSaxHandler.startElement(OdfFileSaxHandler.java:121) I will keep this issue in mind, when I move the changes back to the main repository.
        Hide
        Svante Schubert added a comment -

        Finally I got an exception: As it seems RDFa related, I am curious if it helps if I change the RDFa implementation back...

        java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.util.zip.ZipCoder.getBytes(ZipCoder.java:80)
        at java.util.zip.ZipFile.getEntry(ZipFile.java:306)
        at java.util.jar.JarFile.getEntry(JarFile.java:226)
        at java.util.jar.JarFile.getJarEntry(JarFile.java:209)
        at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:832)
        at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:810)
        at sun.misc.URLClassPath.findResource(URLClassPath.java:176)
        at java.net.URLClassLoader$2.run(URLClassLoader.java:551)
        at java.net.URLClassLoader$2.run(URLClassLoader.java:549)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findResource(URLClassLoader.java:548)
        at java.lang.ClassLoader.getResource(ClassLoader.java:1139)
        at java.lang.ClassLoader.getSystemResource(ClassLoader.java:1259)
        at java.lang.ClassLoader.getSystemResourceAsStream(ClassLoader.java:1362)
        at java.lang.Class.getResourceAsStream(Class.java:2086)
        at javax.xml.stream.SecuritySupport$4.run(SecuritySupport.java:92)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.xml.stream.SecuritySupport.getResourceAsStream(SecuritySupport.java:87)
        at javax.xml.stream.FactoryFinder.findJarServiceProvider(FactoryFinder.java:322)
        at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:289)
        at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:213)
        at javax.xml.stream.XMLOutputFactory.newInstance(XMLOutputFactory.java:129)
        at net.rootdev.javardfa.Parser.<init>(Parser.java:43)
        at org.odftoolkit.odfdom.pkg.rdfa.RDFaParser.<init>(RDFaParser.java:63)
        at org.odftoolkit.odfdom.pkg.rdfa.DOMRDFaParser.<init>(DOMRDFaParser.java:44)
        at org.odftoolkit.odfdom.pkg.rdfa.DOMRDFaParser.createInstance(DOMRDFaParser.java:38)
        at org.odftoolkit.odfdom.pkg.OdfFileDom.updateInContentMetadataCache(OdfFileDom.java:691)
        at org.odftoolkit.odfdom.dom.element.table.TableTableCellElementBase.onInsertNode(TableTableCellElementBase.java:524)
        at org.odftoolkit.odfdom.pkg.OdfElement.onInsertNode(OdfElement.java:478)
        at org.odftoolkit.odfdom.pkg.OdfElement.insertBefore(OdfElement.java:490)
        at org.apache.xerces.dom.NodeImpl.appendChild(Unknown Source)
        at org.odftoolkit.odfdom.pkg.OdfFileSaxHandler.startElement(OdfFileSaxHandler.java:121)

        Show
        Svante Schubert added a comment - Finally I got an exception: As it seems RDFa related, I am curious if it helps if I change the RDFa implementation back... java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.zip.ZipCoder.getBytes(ZipCoder.java:80) at java.util.zip.ZipFile.getEntry(ZipFile.java:306) at java.util.jar.JarFile.getEntry(JarFile.java:226) at java.util.jar.JarFile.getJarEntry(JarFile.java:209) at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:832) at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:810) at sun.misc.URLClassPath.findResource(URLClassPath.java:176) at java.net.URLClassLoader$2.run(URLClassLoader.java:551) at java.net.URLClassLoader$2.run(URLClassLoader.java:549) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findResource(URLClassLoader.java:548) at java.lang.ClassLoader.getResource(ClassLoader.java:1139) at java.lang.ClassLoader.getSystemResource(ClassLoader.java:1259) at java.lang.ClassLoader.getSystemResourceAsStream(ClassLoader.java:1362) at java.lang.Class.getResourceAsStream(Class.java:2086) at javax.xml.stream.SecuritySupport$4.run(SecuritySupport.java:92) at java.security.AccessController.doPrivileged(Native Method) at javax.xml.stream.SecuritySupport.getResourceAsStream(SecuritySupport.java:87) at javax.xml.stream.FactoryFinder.findJarServiceProvider(FactoryFinder.java:322) at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:289) at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:213) at javax.xml.stream.XMLOutputFactory.newInstance(XMLOutputFactory.java:129) at net.rootdev.javardfa.Parser.<init>(Parser.java:43) at org.odftoolkit.odfdom.pkg.rdfa.RDFaParser.<init>(RDFaParser.java:63) at org.odftoolkit.odfdom.pkg.rdfa.DOMRDFaParser.<init>(DOMRDFaParser.java:44) at org.odftoolkit.odfdom.pkg.rdfa.DOMRDFaParser.createInstance(DOMRDFaParser.java:38) at org.odftoolkit.odfdom.pkg.OdfFileDom.updateInContentMetadataCache(OdfFileDom.java:691) at org.odftoolkit.odfdom.dom.element.table.TableTableCellElementBase.onInsertNode(TableTableCellElementBase.java:524) at org.odftoolkit.odfdom.pkg.OdfElement.onInsertNode(OdfElement.java:478) at org.odftoolkit.odfdom.pkg.OdfElement.insertBefore(OdfElement.java:490) at org.apache.xerces.dom.NodeImpl.appendChild(Unknown Source) at org.odftoolkit.odfdom.pkg.OdfFileSaxHandler.startElement(OdfFileSaxHandler.java:121)
        Hide
        Svante Schubert added a comment -

        I tested it on the the latest sources, just used the LoadSaveTest and created a 3.8 ods, which had 79MB of content.xml.
        The test hangs or takes for ages.

        The good news, I tested it on my branch and it works.
        So I have done a fix in the past or the current release has some broken feature.
        No time to merge now, but I looked over OdfPackage and I would start to exchange the existing multi-threaded getInputStream

        /**

        • Gets the InputStream containing whole OdfPackage.
        • @return the ODF package as input stream
        • @throws java.lang.Exception
        • - if the package could not be read
          */
          public InputStream getInputStream() throws Exception {
          final PipedOutputStream os = new PipedOutputStream();
          final PipedInputStream is = new PipedInputStream();

        is.connect(os);

        Thread thread1 = new Thread() {

        @Override
        public void run() {
        try

        { save(os, mBaseURI); }

        catch (Exception e) {
        }
        }
        };

        Thread thread2 = new Thread() {

        @Override
        public void run() {
        try

        { BufferedInputStream bis = new BufferedInputStream(is, StreamHelper.PAGE_SIZE); BufferedOutputStream bos = new BufferedOutputStream(os, StreamHelper.PAGE_SIZE); StreamHelper.transformStream(bis, bos); is.close(); os.close(); }

        catch (Exception ie) {
        }
        }
        };

        thread1.start();
        thread2.start();

        return is;
        }

        With the safer following way:

        /**

        • Gets the InputStream containing whole OdfPackage.
          *
        • @return the ODF package as input stream
        • @throws java.io.IOException
        • - if the package could not be read
          */
          public InputStream getInputStream() throws IOException { ByteArrayOutputStream out = new ByteArrayOutputStream(); save(out, mBaseURI); return new ByteArrayInputStream(out.toByteArray()); }

        Hope that helps!
        Svante

        Show
        Svante Schubert added a comment - I tested it on the the latest sources, just used the LoadSaveTest and created a 3.8 ods, which had 79MB of content.xml. The test hangs or takes for ages. The good news, I tested it on my branch and it works. So I have done a fix in the past or the current release has some broken feature. No time to merge now, but I looked over OdfPackage and I would start to exchange the existing multi-threaded getInputStream /** Gets the InputStream containing whole OdfPackage. @return the ODF package as input stream @throws java.lang.Exception - if the package could not be read */ public InputStream getInputStream() throws Exception { final PipedOutputStream os = new PipedOutputStream(); final PipedInputStream is = new PipedInputStream(); is.connect(os); Thread thread1 = new Thread() { @Override public void run() { try { save(os, mBaseURI); } catch (Exception e) { } } }; Thread thread2 = new Thread() { @Override public void run() { try { BufferedInputStream bis = new BufferedInputStream(is, StreamHelper.PAGE_SIZE); BufferedOutputStream bos = new BufferedOutputStream(os, StreamHelper.PAGE_SIZE); StreamHelper.transformStream(bis, bos); is.close(); os.close(); } catch (Exception ie) { } } }; thread1.start(); thread2.start(); return is; } With the safer following way: /** Gets the InputStream containing whole OdfPackage. * @return the ODF package as input stream @throws java.io.IOException - if the package could not be read */ public InputStream getInputStream() throws IOException { ByteArrayOutputStream out = new ByteArrayOutputStream(); save(out, mBaseURI); return new ByteArrayInputStream(out.toByteArray()); } Hope that helps! Svante
        Hide
        Jacinto Verdaguer added a comment -

        The error occurs with any file 3 MB.

        Tried to update to the latest 0.6 (odfdom-java-0.8.9-incubating.jar, simple-odf-0.8-incubating.jar and xercesImpl-2.9.1.jar) but now not even read the source file.

        Runs the line doc=OdfDocument.loadDocument(new File(fileIn)) and jumps directly to the finally block, no errors display.

        try

        { doc = OdfDocument.loadDocument(new File("C:\tmp\ficheroIn.ods")); OdfFileDom metadom = doc.getMetaDom(); Meta metadata = new Meta(metadom); metadata.setTitle(plantilla.getTitulo()); metadata.setSubject(plantilla.getAsunto()); metadata.setCreator(plantilla.getModificadoPor()); metadata.setDescription(plantilla.getComentarios()); doc.save("C:\tmp\ficheroOut.ods"); }

        catch (Exception ex)

        { plantilla.setErrorProceso(true); }

        finally

        { if(doc != null) doc.close(); }

        My environment is: Windows 7 (64 bits), Java 1.6

        Show
        Jacinto Verdaguer added a comment - The error occurs with any file 3 MB. Tried to update to the latest 0.6 (odfdom-java-0.8.9-incubating.jar, simple-odf-0.8-incubating.jar and xercesImpl-2.9.1.jar) but now not even read the source file. Runs the line doc=OdfDocument.loadDocument(new File(fileIn)) and jumps directly to the finally block, no errors display. try { doc = OdfDocument.loadDocument(new File("C:\tmp\ficheroIn.ods")); OdfFileDom metadom = doc.getMetaDom(); Meta metadata = new Meta(metadom); metadata.setTitle(plantilla.getTitulo()); metadata.setSubject(plantilla.getAsunto()); metadata.setCreator(plantilla.getModificadoPor()); metadata.setDescription(plantilla.getComentarios()); doc.save("C:\tmp\ficheroOut.ods"); } catch (Exception ex) { plantilla.setErrorProceso(true); } finally { if(doc != null) doc.close(); } My environment is: Windows 7 (64 bits), Java 1.6
        Hide
        Svante Schubert added a comment -

        Without an error and a test to reproduce, it is hard to guess. My guess it is a loop..

        My tip:
        1) Update to the latest 0.6 version -> http://incubator.apache.org/odftoolkit/downloads.html
        2) Debug yourself, best adjust an existing test case, or you might add inbetween the lines some Logging/System.err.println

        You might debug/take a look, what happens if no meta.xml exists.

        Show
        Svante Schubert added a comment - Without an error and a test to reproduce, it is hard to guess. My guess it is a loop.. My tip: 1) Update to the latest 0.6 version -> http://incubator.apache.org/odftoolkit/downloads.html 2) Debug yourself, best adjust an existing test case, or you might add inbetween the lines some Logging/System.err.println You might debug/take a look, what happens if no meta.xml exists.

          People

          • Assignee:
            Unassigned
            Reporter:
            Jacinto Verdaguer
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development