Namespace of child element written incorrectly as root namespace

• Key: XALANJ-2219
• Type: Bug
• Status: Resolved
• Resolution: Fixed
• Priority: Major
• Assignee: Brian Minchau
• Reporter: Jesse Glick
• Component/s: Serialization
• Affects Version/s: 2.7, 2.6, 2.5
• Fix Version/s: 2.7.1

Environment

Linux m2 2.6.13-1.1532_FC4 #1 Thu Oct 20 01:30:08 EDT 2005 i686 i686 i386 GNU/Linux
java version "1.5.0_05"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_05-b05)
Java HotSpot(TM) Client VM (build 1.5.0_05-b05, mixed mode, sharing)

Description

Run this class:

---%<---
import java.io.StringReader;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.Result;
import javax.xml.transform.Source;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
import org.w3c.dom.Document;
import org.xml.sax.InputSource;
public class Test {
    private static final String IDENTITY_XSLT_WITH_INDENT =
            "<xsl:stylesheet version='1.0' " +
            "xmlns:xsl='http://www.w3.org/1999/XSL/Transform' " +
            "xmlns:xalan='http://xml.apache.org/xslt' " +
            "exclude-result-prefixes='xalan'>" +
            "<xsl:output method='xml' indent='yes' xalan:indent-amount='4'/>" +
            "<xsl:template match='@*|node()'>" +
            "<xsl:copy>" +
            "<xsl:apply-templates select='@*|node()'/>" +
            "</xsl:copy>" +
            "</xsl:template>" +
            "</xsl:stylesheet>";
    public static void main(String[] args) throws Exception {
        String data = "<root xmlns='root'/>";
        Document doc = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(new InputSource(new StringReader(data)));
        /*
        Document doc = DocumentBuilderFactory.newInstance().newDocumentBuilder().getDOMImplementation().createDocument("root", "root", null);
         */
        doc.getDocumentElement().appendChild(doc.createElementNS("child", "child"));
        Transformer t = TransformerFactory.newInstance().newTransformer(
                new StreamSource(new StringReader(IDENTITY_XSLT_WITH_INDENT)));
        Source source = new DOMSource(doc);
        Result result = new StreamResult(System.out);
        t.transform(source, result);
    }
}
---%<---

Just using plain JDK 5.0 JAXP, I get the expected

---%<---
<?xml version="1.0" encoding="UTF-8"?>
<root xmlns="root">
    <child xmlns="child"/>
</root>
---%<---

If I add Xalan-J to the classpath, I get

---%<---
<?xml version="1.0" encoding="UTF-8"?>
<root xmlns="root">
    <child xmlns="root"/>
</root>
---%<---

Note the incorrect namespace on the child element.

This is true in Xalan 2.5.2, 2.6.0, 2.7.0, and dev builds (xalan-gump-24102005.jar).

Prevents Xalan from being bundled with the NetBeans IDE, as it causes incorrectly written project metadata:

http://www.netbeans.org/issues/show_bug.cgi?id=66563

If you use newTransformer() with no stylesheet the problem goes away (though of course you lose indentation unless it is readded using setOutputProperty). Also if the Document is created in memory rather than parsed (see commented-out code) the problem goes away.

Does not seem to be reproducible in JDK 6, I don't know why.

Inclusion of the bundled Xerces in the classpath does not appear to make any difference.

Comments

[Brian Minchau]

I've modified the code to set the DocumentBuilderFactory to be namespace aware. The problem persisted.

Debugging the code I found that originally the child element had xmlns="child" during its creation, but then we copy namespace nodes from the parent, and we stomp on xmlns='child' with the parents xmlns='root'.

Unlike attributes which can be over-ridden using <xsl:attribute> elements in a stylesheet, the same is not true for namespace nodes. So we should only add namespace nodes that we don't already have on an element, not replace ones that we already have.

This is a bug in Xalan-J in the serializer. I have a patch, it needs to be more thoroughly tested.

[Brian Minchau]

The patch (i.e. hack) that I was thinking of is in the serializer's code to add/update attribute values. It already knows if it is adding a new attribute/value pair or replacing an existing attriubtes value. If it is replacing, check if the name is "xmlns". If it is, don't replace.

There is some fuzzyness in the serializer's code on namespace nodes vs. attribute nodes due to the fact that serialized namespace nodes look like attributes in the output XML.

However, Henry Zongaro wrote me:
<<
I suspect that it's not a serializer bug, but rather a bug in the DOM2DTM implementation of the namespace axis (which is being used under the covers on the xsl:copy). The stylesheet is implementing an identity transform, with the exception that it allows pretty-printing.
>>

[Brian Minchau]

Attaching a patch that changes the behavior of the serializer.

Previously all namespace mappings added to the current element were accepted, possibly replacing the URI that a prefix, or the default namespace maps to.

After the start of a element that is being output, if a mapping of a prefix to a URI it is accepted, possibly over-riding any such mapping that the element would inherit from its ancestors. If yet another mapping of the same prefix is given, it is ignored. Previously the URI would have been updated.

[Brian Minchau]

The patch that I attached, ToStream.patch2.txt, has passed these test targets:
> build smoketest
> build smoketest.xsltc
> build alltest
> build conf
> build conf.xsltc

[Jesse Glick]

I would be happy to test whether this fixes the problem with the IBM JDK 1.5. Unfortunately at the moment I am not able to download that JDK; registration is required and the registration service seems to be broken. Let me know if there is some direct download URL I can use, else I will try again later.

[Jesse Glick]

Figured out I can test without using the IBM JDK. As a control, download Xalan-J 2.7.0 and try to run NB with that in bootcp:

rm -rfv /tmp/test-xalanj-2219; /space/nb/nb50/bin/netbeans --userdir /tmp/test-xalanj-2219 -J-Xbootclasspath/p:/tmp/xalan-j_2_7_0/xalan.jar:/tmp/xalan-j_2_7_0/serializer.jar:/tmp/xalan-j_2_7_0/xercesImpl.jar

As expected, it fails: a dialog appears instructing you that the IDE is using a buggy version of Xalan and then the IDE shuts down.

Using SVN trunk sources of xalan, patched with the above patch:

rm -rfv /tmp/test-xalanj-2219; /space/nb/nb50/bin/netbeans --userdir /tmp/test-xalanj-2219 -J-Xbootclasspath/p:/tmp/xalan/build/xalan.jar:/tmp/xalan/build/serializer.jar:/tmp/xalan-j_2_7_0/xercesImpl.jar

this no longer occurs. The IDE starts up and lets you make a project. However it is still broken, just differently. The project.xml which is generated ought to look like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://www.netbeans.org/ns/project/1">
    <type>org.netbeans.modules.java.j2seproject</type>
    <configuration>
        <data xmlns="http://www.netbeans.org/ns/j2se-project/3">
            <name>JavaApplication1</name>
            <minimum-ant-version>1.6.5</minimum-ant-version>
            <source-roots>
                <root id="src.dir"/>
            </source-roots>
            <test-roots>
                <root id="test.src.dir"/>
            </test-roots>
        </data>
    </configuration>
</project>

However it instead looks like this:

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://www.netbeans.org/ns/project/1">
    <type>org.netbeans.modules.java.j2seproject</type>
    <configuration>
        <data xmlns="http://www.netbeans.org/ns/j2se-project/3">
            <name xmlns="http://www.netbeans.org/ns/project/1">JavaApplication1</name>
            <minimum-ant-version xmlns="http://www.netbeans.org/ns/project/1">1.6.5</minimum-ant-version>
            <source-roots xmlns="http://www.netbeans.org/ns/project/1">
                <root xmlns="http://www.netbeans.org/ns/j2se-project/3" id="src.dir"/>
            </source-roots>
            <test-roots xmlns="http://www.netbeans.org/ns/project/1">
                <root xmlns="http://www.netbeans.org/ns/j2se-project/3" id="test.src.dir"/>
            </test-roots>
        </data>
    </configuration>
</project>

Note the unwanted xmlns attrs inside <data>. If it is not obvious what is wrong please let me know and I will try to come up with an additional test case.

[Jesse Glick]

As an aside, note also the lack of a newline between the XML declaration and the document element. This formatting bug is also visible in the original test case I attached:

<?xml version="1.0" encoding="UTF-8"?><root xmlns="root">
    <child xmlns="child"/>
</root>

[Jesse Glick]

By the way, the Sun JDK's version of Xalan does not appear to suffer from this issue, so the version in src.zip might be used as inspiration. However I am not sure if the lack of this bug is due to an actual fix in the JDK's version of the code, or to using XSLTC rather than the interpreter.

[Brian Minchau]

Jesse,
the lack of indentation after the xml header is not a bug. It is possible that the serialized XML will be used as an external general parsed entity and included elsewhere. Where it is included it my be next to text, and the extra newline would be appended to that text, which would be incorrect. To avoid such problems there is no longer a newline after the xml header.

Inside the document element (in your case <root>) it is OK to indent.

This indentation issue is not a bug.

- Brian

[Henry Zongaro]

I have reviewed the patch, and I believe it resolves the problem.

[Brian Minchau]

The patch was committed to the code repository.

A related but separate bug was found by Henry Zongaro, for which I opened xalanj-2302.

[Brian Minchau]

I may have been a little hasty in resolving this issue as "fixed". However the original problem, as reported, is fixed.

I am aware that NetBeans has code in it checking for a faulty version of Xalan, and gives you this message:
"a dialog appears instructing you that the IDE is using a buggy version of Xalan and then the IDE shuts down."

So with the fix in this issue you get around this but on to the next problem. I'd prefer that you (Jesse) open a new issue for that one. You've given the expected result, and the actual result. Your first testcase was great! Another one for your second problem would be great too. Without a testcase we can't fix your second bug.

- Brian Minchau

[Brian Minchau]

Patch4 is an update to the previous patch reviewed by Henry Z.
An obscure test failed with XSLTC, so I changed the was_added flag in the
patch to be set to true originally. Logical is like before, but it waits for failure from
addAttributeAlways rather than assumes failure.

[Christine Li]

I have reviewed the reworked patch4. It looks good and I belive it resolves the problem

[Brian Minchau]

patch4 has a regression for test build target alltest.conf.xsltc for the trax.sax flavor.

Patch needs to be reworked.

[Brian Minchau]

Yash Talwar found the regression (thanks a lot Yash !) ... so Brian M. will rework the patch.

[Brian Minchau]

Per the JIRA meeting on Oct 16, 2005, the patch in this issue needs to be reversed and taken out of the code base. Somehow it seems that NamespaceMappings.java in org.apache.xml.serializer may already have the fix in it. This needs to be reviewed by Brian M.

[Brian Minchau]

Attaching patch5 which both reverses patch4 to ToStream.java and undoes that fix (and the regressions it caused), but also adds a better fix for this problem, and that fix is in NamespaceMappings.java

[Brian Minchau]

patch5 shows no regressions with these test targets:
build smoketest
build smoketest.xsltc
build alltest
build conf
build conf.xsltc

[Yash Talwar]

I have reviewed the changes made by Brian in patch jira2219.patch5.txt. The changes look good to me.
I approve.

[Brian Minchau]

I just applied patch5 to the code base.
From Jesse Glick's point of view either patch fixes his problem,
but patch5 is just "better".

[Brian Minchau]

Peter Fleischer reports that this bug is not fixed in Xalan-J 2.7.1.
I'll have a look
 - Brian

[Brian Minchau]

Would the originator of this issue please verify that this issue is fixed in the 2.7.1 release, by adding a comment to this issue, so that we can close this issue.

A lack of response by February 1, 2008 will be taken as consent that we can close this resolved issue.

Regards,
Brian Minchau

[Brian Minchau]

My previous comment, asking originators to verify that the issue was fixed in 2.7.1 is a blanket comment that I blindly added to all issues in resolved state just before 2.7.1 was released. So take it with a grain of salt.

I'm still going to have another look at this one.
- Brian