We do have complex types like bag of maps and bag of bags and one or two levels of nesting. But I assume you have way more nesting than we do. Does that matter though as what is written to TFile is just bytes for both key and value?
We use lzo. It would be good to try gz and see if the problem is with lzo for you.
2013-09-24 21:10:21,289 INFO [main] com.hadoop.compression.lzo.GPLNativeCodeLoader: Loaded native gpl library
2013-09-24 21:10:21,291 INFO [main] com.hadoop.compression.lzo.LzoCodec: Successfully loaded & initialized native-lzo library
2013-09-24 21:10:21,293 INFO [main] org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.lzo_deflate]
I don't think hadoop version should matter as we had hadoop 1.x till mid 2012.