The XA transaction timeout feature was added in
DERBY-2432. It introduced a new property "derby.jdbc.xaTransactionTimeout" which defines a default value of XA transaction timeout if it was not specfied in a call to XAResource.setTransactionTimeout. The property might be specified also as a database property.
The documentation should be probably added to 'Derby properties' chapter to "Tuning Derby" guide.