I think that it's important enough to clarify:
When hsync returns the DNs have all gotten the data and the request to flush and sync, but they are not necessarily finished with it.
It's NOT the "proper" fsync semantics then. In fact, this is even worse than hflush in branch-1, which acks after data is flushed to OS cache (written to FileOutputStream). i.e., HBase with Hadoop 1.x can survive kill -9 all DNs at the same time without data loss, while this hsync implementation cannot.
The same, BTW, is currently (without my patch) true to hflush. When hflush returns the DNs have received the data in their buffers, but they may not be done flushing the data to the OS buffers.
This is an unfortunate side-effect of HDS-265 that makes branch-1 hflush more durable (which is not enough) than hsync/hflush in trunk (and all post 0.21 code).
If we wanted to fix this (which would involve doing the flushOrSync in the PacketResponder if the request is a client request, which inconveniently would also serialize the flush/syncs across replicas), this should be a different jira, I think.
Did you file a JIRA Lars? BTW, instead of doing the sync in the responder (ack thread), an alternative implementation could use a conditional variable to signal sync done in the data thread..., which I think, is a small price to pay for a parallel sync with the correct hsync semantics.