[ASTERIXDB-1400] Loading of non-empty datasets should be supported

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: *DB - AsterixDB, AQL - Translator AQL, OTH - Other, STO - Storage
• Assignee: Xikui Wang
• Reporter: Michael J. Carey
• Created: 14/Apr/16 23:36
• Updated: 02/Aug/19 22:26

Description

From a user model perspective, we should have a "load more" capability so that users can add data into an existing dataset using load-like DDL. Under the hood this can essentially be an insert-into-from-external-files feature, i.e., it doesn't have to be optimized yet, beyond being executed as a single Hyracks job that does a series of inserts (into the dataset and its associated indexes) - as from a user's perspective today it is REALLY a pain to not be able to add more data after an initial load! As a result we see folks having to do work around like loading into some other dataset and then doing an insert-into, or abusing feeds, or.... Let's fix this! It can't be super-hard, as it would essentially operate like a short-lived datafeed under the hood.

Issue Links

• is related to: ASTERIXDB-1363 Internal error when accidentally loading the same dataset 2x