I profiled an expensive CAS deserialization and found that 46% of the time was in CASImpl.ll_getFSForRef (the method that creates a FeatureStructure Java object for a CAS FS). All those calls were coming from deserializing arrays (of which this particular CAS has many).
It is unnecessary to create FeatureStructure Java objects here. For non-array FSs, XmiCasDeserializer uses low-level CAS APIs in order to avoid this overhead. But for arrays, it currently does not use the low-level APIs.