How does that work with containers?
That depends on the container. Generally, we can always read the list itself, since we read the type header and size. However, if some of the values are invalid, things get a little icky. In a list, you'd get one or more null fields. In a set, you'd get exactly one null field, thus reducing the size reported on the wire. In a map, if the key was null, then the entire entry would go away (at least in java, which doesn't allow null in HashMap keys); if the value was null, only that part of the entry would be null.
The idea of an annotation on enum and union types that indicates whether an unexpected value should trigger a validation error is appealing to me.
We already have this annotation - it's called "required". If it's an unexpected value, you'll get null back, and if that field is required, that's a validation error. If you don't want the validation error, don't make the field required. Can you imagine a scenario where you'd want a field of type enum to be required yet not care if the value is recognizable?
I don't think that "unset" and "unexpected value" have a meaningful difference to any user of Thrift, since in either case the "value" will just appear to be a null. If we treat them differently, then perhaps "unset" could be determined whether or not the value is null. However, I don't really know how you would make use of this new feature other than in debugging a null value that occurred unexpectedly. Also, trying to achieve this duality would eliminate some optimizations we've made to the memory footprint of structs.
I should also point out that in Java, our enums are actual enums, not ints. This means that if we don't recognize the value's id on the wire, we literally have no way other than null to represent that it was unrecognized.