Details
Major Description
BytesToBytesMap current encodes key/value data in the following format:
8B key length, key data, 8B value length, value data
UnsafeExternalSorter, on the other hand, encodes data this way:
4B record length, data
As a result, we cannot pass records encoded by BytesToBytesMap directly into UnsafeExternalSorter for sorting. However, if we rearrange data slightly, we can then pass the key/value records directly into UnsafeExternalSorter:
4B key+value length, 4B key length, key data, value data
Attachments
Issue Links
- links to
