In general it good, but I would still advertise the SingularMatrixException in the DecompositionSolver interface with a @throws clause.
For each decomposition, I would specify if it can compute a pseudo-inverse or will throw an exception in case of a singular matrix.
The reason being that a user might just use an interface and may not know explicitly the used implementation, thus must protect itself against anything that can happen, including the SingularMatrixException.
btw. I would also add the specific inverse behavior, as mentioned before, to the class javadoc of each decomposition, as this one will be more often read as the one from the embedded DecompositionSolver imho.