Uploading a patch that fixes the issue.
Turns out buggy use of DataInputBuffer.reset() and DataInputBuffer.getLength() was leading to integer overflow at these considerable large values of io.sort.mb.
getLength() returns the size of the entire buffer, and not just the remaining part of the buffer. The offending code assumes otherwise leading to this issue.
For instance, if a key is at position 1,224,906,830 extending to 1,224,906,868. The length of this key should be set to 38 - the code instead sets it to 1,224,906,868. The data buffer interprets this key to end at 2,449,813,698 which is bigger than 2^31-1 leading to a negative value!!
For something like this to happen, the starting position of the key should be large ~ > (2^31 - 1 - key_size)/2 = 1073741823.5 - key_size/2.
The reported 1280 MB is larger than this, and hence we see the issue.
The patch fixes the use of reset() and getLength() as required, and also updates the javadoc of getLength().
Verified that the patch fixes the problem for the same cases as in the description. Not sure how to write a test for this.