In my code I prefer throwing the first one. There is one exception when a latter Throwable occurrs and it is an Error and the former not.
In my opinion this is the Throwable you want to see.
Another approach is to throw something like a MultiIOException containing all occurred exceptions.
I agree that this all is not a real pleasure, but better than leaving resources open that can be closed successfully.
I've written a MultiOutputStream yesterday: https://github.com/fabian-barney/Utils/blob/master/utils/src/com/barney4j/utils/io/MultiOutputStream.java
I am not sure for myself that I made the right decision here.