So, one possibility I am looking at for the data [<topic_data_struct>] is to use ByteArrayOutputStream and ObjectOutputStream to make the List[WiredTopic] get into the byte buffer. My concern here though is non JVM clients not being able to support it.
Another option is making this new format Thrift based (or protobuffer, etc) though I also appreciate and understand that adding yet another dependency yet it allows for language independence for what is now a complex data structure (not the simple 3 fields that exist now). The entire format could be a Thrift object and the serialized bytes become a single put into the byte buffer
The last option I have considered is too basically loop through the data structures and increase the ByteBuffer position structure. So after the ack timeout we could store some hint in the byte buffer to the  of topic -> [partition,message] e.g. we have 2 topics being published too with one of those topics having 2 partitions with a message se and the other having 3 paritions/messageet. So we could create some "hint" to know the number of topics and each of the counts of paritions/message for each topic (in this case 2,2,3) and "put" the topic1, partitionA, messageA, partitionB,messageB, topic2, partitionC, messageC, partitionD,messageD, partitionE,messageE. The draw back here is some nuance complexity (bordering on esoteric) to take the object model break it out, store it and then pulling the stored value (based on the hint so we know what position is topic and which are partition. The "hint" could be a delimited string maybe (if this is the approach that is adopted) count of topics and for each topic then the count of partition for those topics. 2,2,3 split on ,  is the count of topics  is the count of partition/message for topic1 and  is the count of partition/message for topic 2
might be some other options here? I am missing something/over complicating? my preference is the thrift approach but I appreciate the "hint" approach also and I would be quite alright with that too.... it works with no additive dependency drawback is a tad harder to have client code ("driver") adoption