The serializer's code to set values from xsl:output attributes is brittle.

Setting a writer or OutputStream involves internal "init" methods that take java.util.Properties objects. Setting an OutputStream should not need to do all that. The only interaction with the properties is the encoding, which would be used in the creation of the java.io.Writer that wraps the OutputStream, but that is about it.

The code is also brittle with respect to whether the values set are default values or explicit values,
say from a stylesheet or from JAXP APIs.

In the long run many methods on the serializer's XSLOutputAttributes interface should go away.
In their place a more generic methods should exist:

public void setOutputPropertyDefault(String name, String val);
public void setOutputProperty(String name, String val);

public String getOutputPropertyDefault(String name);
public String getOutputProperty(String name);

Through these methods one can set/get a default or non-default property.