Major Description
The XMLDSIG specification requires a straight big-endian binary encoding of r and s:
The signature value consists of the base64 encoding of the concatenation of two octet-streams that respectively result from the octet-encoding of the values r and s in that order. Integer to octet-stream conversion must be done according to the I2OSP operation defined in the RFC 3447 [PKCS1] specification with a l parameter equal to 20.
However, the current code does not zero-pad the values to 20 bytes, leading to Signature Lenght Incorrect messages during verify. Moreover, it allocates rawSigBuf based on bit sizes, but writes into it with byte granularity. If both r and s are (say) 156 bits, they are still 20 bytes each, but rawSigBuf is allocated 39 bytes only, one less than necessary.
The attached patch made signature generation with 160-bit q-s work reliably for me. I'd be interested to hear how bigger signatures should be handled according to the DSIG specification; I tried to find out but failed.