[OAK-7389] Mongo/FileBlobStore does not update timestamp for already existing blobs

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.2.14, 1.4.20, 1.8.2, 1.6.11
• Fix Version/s: 1.9.0, 1.10.0, 1.2.30, 1.4.23, 1.6.14, 1.8.7
• Component/s: blob
• Assignee: Amit Jain
• Reporter: Amit Jain
• Created: 05/Apr/18 06:44
• Updated: 07/May/20 12:10
• Resolved: 06/Apr/18 09:01

Description

MongoBlobStore uses uses the insert call and ignores any exceptions which means any existing value won't be updated.

    @Override
    protected void storeBlock(byte[] digest, int level, byte[] data) throws IOException {
        String id = StringUtils.convertBytesToHex(digest);
        cache.put(id, data);
        // Check if it already exists?
        MongoBlob mongoBlob = new MongoBlob();
        mongoBlob.setId(id);
        mongoBlob.setData(data);
        mongoBlob.setLevel(level);
        mongoBlob.setLastMod(System.currentTimeMillis());
        // TODO check the return value
        // TODO verify insert is fast if the entry already exists
        try {
            getBlobCollection().insertOne(mongoBlob);
        } catch (DuplicateKeyException e) {
            // the same block was already stored before: ignore
        } catch (MongoException e) {
            if (e.getCode() == DUPLICATE_KEY_ERROR_CODE) {
                // the same block was already stored before: ignore
            } else {
                throw new IOException(e.getMessage(), e);
            }
        }
    }

 FileBlobStore also returns if there's a file already existing without updating the timestamp

    @Override
    protected synchronized void storeBlock(byte[] digest, int level, byte[] data) throws IOException {
        File f = getFile(digest, false);
        if (f.exists()) {
            return;
        }
        .........

The above would cause data loss in DSGC if there are updates to the blob blocks which are re-surrected (stored again at the time of DSGC) because the timestamp would never have been modified.

 

cc/ tmueller, mreutegg, chetanm, catholicon

Issue Links

• is related to: OAK-7392 Use upsert in MongoBlobStore to update timestamps for existing blobs