Harmony
  1. Harmony
  2. HARMONY-3850

[classlib][encoding][eut] testRootElementAndDTDDescriber fails due to encoding issues

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Classlib
    • Labels:
      None

      Description

      tests
      testContentDescription
      testRootElementAndDTDDescriber
      fail on DRLVM in OPT JET and interpreter modes.
      And they also fail on J9+Harmony classlib.

      See parent issue for the output example.

        Issue Links

          Activity

          Hide
          Vasily Zakharov added a comment -

          Problem reproduced, investigating.

          Show
          Vasily Zakharov added a comment - Problem reproduced, investigating.
          Hide
          Vasily Zakharov added a comment -

          The problem at IContentTypeManagerTest.testContentDescription ocurrs at the following line (IContentTypeManagerTest.java:447):

          assertTrue("2.4d" + sufix, text || IContentDescription.BOM_UTF_16BE == description.getProperty(IContentDescription.BYTE_ORDER_MARK));

          Note that IContentDescription.BOM_UTF_16BE is defined as

          public final static byte[] BOM_UTF_16BE =

          {(byte) 0xFE, (byte) 0xFF}

          ;

          Comparing arrays with == operator seems to be the cause of the problem. However, deeper investigation is necessary to make sure this is really the cause of the problem and to clarify why the problem doesn't occur on RI.

          Show
          Vasily Zakharov added a comment - The problem at IContentTypeManagerTest.testContentDescription ocurrs at the following line (IContentTypeManagerTest.java:447): assertTrue("2.4d" + sufix, text || IContentDescription.BOM_UTF_16BE == description.getProperty(IContentDescription.BYTE_ORDER_MARK)); Note that IContentDescription.BOM_UTF_16BE is defined as public final static byte[] BOM_UTF_16BE = {(byte) 0xFE, (byte) 0xFF} ; Comparing arrays with == operator seems to be the cause of the problem. However, deeper investigation is necessary to make sure this is really the cause of the problem and to clarify why the problem doesn't occur on RI.
          Hide
          Vasily Zakharov added a comment -

          The previous assumption was wrong. Investigation shows that == operator is used correctly, but description.getProperty() returns IContentDescription.BOM_UTF_16LE instead of IContentDescription.BOM_UTF_16BE like it should, so it seems that a real UTF decoding problem is in place.

          Show
          Vasily Zakharov added a comment - The previous assumption was wrong. Investigation shows that == operator is used correctly, but description.getProperty() returns IContentDescription.BOM_UTF_16LE instead of IContentDescription.BOM_UTF_16BE like it should, so it seems that a real UTF decoding problem is in place.
          Hide
          Vasily Zakharov added a comment -

          Issue 1 investigated. The problem is in InputStream that handles UTF-16 encoding differently than RI. At the following point BOM_UTF_16LE is returned instead of BOM_UTF_16BE because incorrect bytes are read from the stream:

          at org.eclipse.core.internal.content.TextContentDescriber.getByteOrderMark(TextContentDescriber.java:65)
          at org.eclipse.core.internal.content.XMLContentDescriber.describe(XMLContentDescriber.java:35)
          at org.eclipse.core.internal.content.ContentType.describe(ContentType.java:161)
          at org.eclipse.core.internal.content.ContentType.internalGetDescriptionFor(ContentType.java:459)
          at org.eclipse.core.internal.content.ContentTypeCatalog.getDescriptionFor(ContentTypeCatalog.java:314)
          at org.eclipse.core.internal.content.ContentTypeCatalog.getDescriptionFor(ContentTypeCatalog.java:319)
          at org.eclipse.core.internal.content.ContentTypeMatcher.getDescriptionFor(ContentTypeMatcher.java:86)
          at org.eclipse.core.tests.runtime.content.IContentTypeManagerTest.getDescriptionFor(IContentTypeManagerTest.java:101)
          at org.eclipse.core.tests.runtime.content.IContentTypeManagerTest.testContentDescription(IContentTypeManagerTest.java:442)

          Issue HARMONY-4135 is filed for this problem.

          Show
          Vasily Zakharov added a comment - Issue 1 investigated. The problem is in InputStream that handles UTF-16 encoding differently than RI. At the following point BOM_UTF_16LE is returned instead of BOM_UTF_16BE because incorrect bytes are read from the stream: at org.eclipse.core.internal.content.TextContentDescriber.getByteOrderMark(TextContentDescriber.java:65) at org.eclipse.core.internal.content.XMLContentDescriber.describe(XMLContentDescriber.java:35) at org.eclipse.core.internal.content.ContentType.describe(ContentType.java:161) at org.eclipse.core.internal.content.ContentType.internalGetDescriptionFor(ContentType.java:459) at org.eclipse.core.internal.content.ContentTypeCatalog.getDescriptionFor(ContentTypeCatalog.java:314) at org.eclipse.core.internal.content.ContentTypeCatalog.getDescriptionFor(ContentTypeCatalog.java:319) at org.eclipse.core.internal.content.ContentTypeMatcher.getDescriptionFor(ContentTypeMatcher.java:86) at org.eclipse.core.tests.runtime.content.IContentTypeManagerTest.getDescriptionFor(IContentTypeManagerTest.java:101) at org.eclipse.core.tests.runtime.content.IContentTypeManagerTest.testContentDescription(IContentTypeManagerTest.java:442) Issue HARMONY-4135 is filed for this problem.
          Hide
          Vasily Zakharov added a comment -

          Issue 1 is resolved, as HARMONY-4135 is no longer reproducible on the current build, clearly after nio.char code was replaced.

          Show
          Vasily Zakharov added a comment - Issue 1 is resolved, as HARMONY-4135 is no longer reproducible on the current build, clearly after nio.char code was replaced.
          Hide
          Vasily Zakharov added a comment -

          Issue 2 investigated. The problem is filed as HARMONY-4196. It is caused by the exception at the following point:

          java.io.UnsupportedEncodingException
          at java.io.InputStreamReader.<init>(InputStreamReader.java:103)
          at org.apache.xerces.impl.XMLEntityManager.createReader(Unknown Source)
          at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
          at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)
          at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
          at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
          at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
          at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
          at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
          at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)
          at org.eclipse.core.internal.content.XMLRootHandler.parseContents(XMLRootHandler.java:160)
          at org.eclipse.core.runtime.content.XMLRootElementContentDescriber.checkCriteria(XMLRootElementContentDescriber.java:74)
          at org.eclipse.core.runtime.content.XMLRootElementContentDescriber.describe(XMLRootElementContentDescriber.java:108)
          at org.eclipse.core.internal.content.ContentType.describe(ContentType.java:162) <<<<< !!! at this point this exception is caught and discarded !!!
          at org.eclipse.core.internal.content.ContentTypeCatalog.collectMatchingByContents(ContentTypeCatalog.java:195)
          at org.eclipse.core.internal.content.ContentTypeCatalog.internalFindContentTypesFor(ContentTypeCatalog.java:374)
          at org.eclipse.core.internal.content.ContentTypeCatalog.internalFindContentTypesFor(ContentTypeCatalog.java:430)
          at org.eclipse.core.internal.content.ContentTypeCatalog.getDescriptionFor(ContentTypeCatalog.java:314)
          at org.eclipse.core.internal.content.ContentTypeCatalog.getDescriptionFor(ContentTypeCatalog.java:330)
          at org.eclipse.core.internal.content.ContentTypeMatcher.getDescriptionFor(ContentTypeMatcher.java:86)
          at org.eclipse.core.tests.runtime.content.IContentTypeManagerTest.testRootElementAndDTDDescriber(IContentTypeManagerTest.java:1200)

          Show
          Vasily Zakharov added a comment - Issue 2 investigated. The problem is filed as HARMONY-4196 . It is caused by the exception at the following point: java.io.UnsupportedEncodingException at java.io.InputStreamReader.<init>(InputStreamReader.java:103) at org.apache.xerces.impl.XMLEntityManager.createReader(Unknown Source) at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source) at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source) at org.eclipse.core.internal.content.XMLRootHandler.parseContents(XMLRootHandler.java:160) at org.eclipse.core.runtime.content.XMLRootElementContentDescriber.checkCriteria(XMLRootElementContentDescriber.java:74) at org.eclipse.core.runtime.content.XMLRootElementContentDescriber.describe(XMLRootElementContentDescriber.java:108) at org.eclipse.core.internal.content.ContentType.describe(ContentType.java:162) <<<<< !!! at this point this exception is caught and discarded !!! at org.eclipse.core.internal.content.ContentTypeCatalog.collectMatchingByContents(ContentTypeCatalog.java:195) at org.eclipse.core.internal.content.ContentTypeCatalog.internalFindContentTypesFor(ContentTypeCatalog.java:374) at org.eclipse.core.internal.content.ContentTypeCatalog.internalFindContentTypesFor(ContentTypeCatalog.java:430) at org.eclipse.core.internal.content.ContentTypeCatalog.getDescriptionFor(ContentTypeCatalog.java:314) at org.eclipse.core.internal.content.ContentTypeCatalog.getDescriptionFor(ContentTypeCatalog.java:330) at org.eclipse.core.internal.content.ContentTypeMatcher.getDescriptionFor(ContentTypeMatcher.java:86) at org.eclipse.core.tests.runtime.content.IContentTypeManagerTest.testRootElementAndDTDDescriber(IContentTypeManagerTest.java:1200)
          Hide
          Vasily Zakharov added a comment -

          Here's more precise stack (not spoiled by inserted debugging lines) for Issue 2:

          java.io.UnsupportedEncodingException
          at java.io.InputStreamReader.<init>(InputStreamReader.java:104)
          at org.apache.xerces.impl.XMLEntityManager.createReader(Unknown Source)
          at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
          at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)
          at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
          at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
          at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
          at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
          at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
          at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)
          at org.eclipse.ant.internal.core.contentDescriber.AntHandler.parseContents(AntHandler.java:130)
          at org.eclipse.ant.internal.core.contentDescriber.AntBuildfileContentDescriber.checkCriteria(AntBuildfileContentDescriber.java:60)
          at org.eclipse.ant.internal.core.contentDescriber.AntBuildfileContentDescriber.describe(AntBuildfileContentDescriber.java:96)
          at org.eclipse.core.internal.content.ContentType.describe(ContentType.java:161)
          (at this point (ContentType.java:172) the exception is caught and discarded )
          at org.eclipse.core.internal.content.ContentTypeCatalog.collectMatchingByContents(ContentTypeCatalog.java:190)
          at org.eclipse.core.internal.content.ContentTypeCatalog.internalFindContentTypesFor(ContentTypeCatalog.java:360)
          at org.eclipse.core.internal.content.ContentTypeCatalog.internalFindContentTypesFor(ContentTypeCatalog.java:407)
          at org.eclipse.core.internal.content.ContentTypeCatalog.getDescriptionFor(ContentTypeCatalog.java:304)
          at org.eclipse.core.internal.content.ContentTypeCatalog.getDescriptionFor(ContentTypeCatalog.java:318)
          at org.eclipse.core.internal.content.ContentTypeMatcher.getDescriptionFor(ContentTypeMatcher.java:86)
          at org.eclipse.core.tests.runtime.content.IContentTypeManagerTest.testRootElementAndDTDDescriber(IContentTypeManagerTest.java:1200)

          Show
          Vasily Zakharov added a comment - Here's more precise stack (not spoiled by inserted debugging lines) for Issue 2: java.io.UnsupportedEncodingException at java.io.InputStreamReader.<init>(InputStreamReader.java:104) at org.apache.xerces.impl.XMLEntityManager.createReader(Unknown Source) at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source) at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source) at org.eclipse.ant.internal.core.contentDescriber.AntHandler.parseContents(AntHandler.java:130) at org.eclipse.ant.internal.core.contentDescriber.AntBuildfileContentDescriber.checkCriteria(AntBuildfileContentDescriber.java:60) at org.eclipse.ant.internal.core.contentDescriber.AntBuildfileContentDescriber.describe(AntBuildfileContentDescriber.java:96) at org.eclipse.core.internal.content.ContentType.describe(ContentType.java:161) (at this point (ContentType.java:172) the exception is caught and discarded ) at org.eclipse.core.internal.content.ContentTypeCatalog.collectMatchingByContents(ContentTypeCatalog.java:190) at org.eclipse.core.internal.content.ContentTypeCatalog.internalFindContentTypesFor(ContentTypeCatalog.java:360) at org.eclipse.core.internal.content.ContentTypeCatalog.internalFindContentTypesFor(ContentTypeCatalog.java:407) at org.eclipse.core.internal.content.ContentTypeCatalog.getDescriptionFor(ContentTypeCatalog.java:304) at org.eclipse.core.internal.content.ContentTypeCatalog.getDescriptionFor(ContentTypeCatalog.java:318) at org.eclipse.core.internal.content.ContentTypeMatcher.getDescriptionFor(ContentTypeMatcher.java:86) at org.eclipse.core.tests.runtime.content.IContentTypeManagerTest.testRootElementAndDTDDescriber(IContentTypeManagerTest.java:1200)
          Hide
          Vladimir Beliaev added a comment -

          The issues is still reproducible on M4 release:

          testRootElementAndDTDDescriber failes with:

          5.1 expected:<org.eclipse.core.tests.runtime.root-element> but was:<org.eclipse.core.runtime.xml>

          junit.framework.AssertionFailedError: 5.1 expected:<org.eclipse.core.tests.runtime.root-element> but was:<org.eclipse.core.runtime.xml>
          at org.eclipse.core.tests.runtime.content.IContentTypeManagerTest.testRootElementAndDTDDescriber(IContentTypeManagerTest.java:1203)
          at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
          at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
          at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
          at org.eclipse.test.CoreTestApplication.runTests(CoreTestApplication.java:35)
          at org.eclipse.test.CoreTestApplication.run(CoreTestApplication.java:31)
          at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
          at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethod(EclipseAppContainer.java:533)
          at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:155)
          at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:106)
          at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:76)
          at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:363)
          at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:176)
          at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
          at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:504)
          at org.eclipse.equinox.launcher.Main.basicRun(Main.java:443)
          at org.eclipse.equinox.launcher.Main.run(Main.java:1169)
          at org.eclipse.equinox.launcher.Main.main(Main.java:1144)
          at org.eclipse.core.launcher.Main.main(Main.java:30)

          So it is the same symptom as it was.

          I recommend:
          1. doing this JIRA as a separate bug, not a sub-task
          2. change its summary to: [classlib][eut] testRootElementAndDTDDescriber fails due to encoding issues
          3. close H3343 as FIXED (because it was a complex JIRA and set of its issues were fixed)

          Thanks
          Vladimir Beliaev

          Show
          Vladimir Beliaev added a comment - The issues is still reproducible on M4 release: testRootElementAndDTDDescriber failes with: 5.1 expected:<org.eclipse.core.tests.runtime.root-element> but was:<org.eclipse.core.runtime.xml> junit.framework.AssertionFailedError: 5.1 expected:<org.eclipse.core.tests.runtime.root-element> but was:<org.eclipse.core.runtime.xml> at org.eclipse.core.tests.runtime.content.IContentTypeManagerTest.testRootElementAndDTDDescriber(IContentTypeManagerTest.java:1203) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java) at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354) at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206) at org.eclipse.test.CoreTestApplication.runTests(CoreTestApplication.java:35) at org.eclipse.test.CoreTestApplication.run(CoreTestApplication.java:31) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java) at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethod(EclipseAppContainer.java:533) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:155) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:106) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:76) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:363) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:176) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:504) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:443) at org.eclipse.equinox.launcher.Main.run(Main.java:1169) at org.eclipse.equinox.launcher.Main.main(Main.java:1144) at org.eclipse.core.launcher.Main.main(Main.java:30) So it is the same symptom as it was. I recommend: 1. doing this JIRA as a separate bug, not a sub-task 2. change its summary to: [classlib] [eut] testRootElementAndDTDDescriber fails due to encoding issues 3. close H3343 as FIXED (because it was a complex JIRA and set of its issues were fixed) Thanks Vladimir Beliaev

            People

            • Assignee:
              Unassigned
              Reporter:
              George Timoshenko
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:

                Development