Details
-
Task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Currently, we compute a diff between the current partition and an already registered partition when a partition has already been registered in Hive. This is done by calling getPartition() on the Hive metastore client, which can be expensive. Since no time-varying attributes are stored in a Hive partition, diff computation (and getPartition() call) can be skipped when a partition already exists.
Attachments
Issue Links
- links to