When running a sparksql query like this (at least on a json dataset)
The rows returned lose the nested structure. I receive a row like
I personally think this is confusing especially when programmatically building and executing queries and then parsing it to find your data in a new structure. I could understand how that's less desirable in some situations, but you could get around it by supporting 'as'. If you wanted to skip the nested structure simply write.
meta_data.name as name