Details
Minor Description
For data stored as a Hive-style partitioned table, data files should only live in leaf partition directories.
For example, the following directory layout is illegal:
.
├── _SUCCESS
├── b=0
│ ├── c=0
│ │ └── part-r-00004.gz.parquet
│ └── part-r-00004.gz.parquet
└── b=1
├── c=1
│ └── part-r-00008.gz.parquet
└── part-r-00008.gz.parquet
For now, we give an unintuitive error message like this:
Conflicting partition column names detected: ArrayBuffer(b, c) ArrayBuffer(b)
This should be improved.
