[HADOOP-16502] Add fsck to S3A tests where additional diagnosis is needed

• Type: Sub-task
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: fs/s3
• Target Version/s: 3.5.0
• Assignee: Unassigned
• Reporter: Gabor Bota
• Created: 09/Aug/19 09:34
• Updated: 04/Jan/24 11:21

Description

Extend org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore#testPruneTombstoneUnderTombstone

    // the child2 entry is still there, though it's now orphan (the store isn't
    // meeting the rule "all entries must have a parent which exists"
    getFile(child2);

    + // todo create a raw fs
    + S3GuardFsck fsck = new S3GuardFsck(rawFs, ms);

    // a full prune will still find and delete it, as this
    // doesn't walk the tree
    getDynamoMetadataStore().prune(PruneMode.ALL_BY_MODTIME,
        now + MINUTE);

Extend org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore#testPutFileDeepUnderTombstone:

    // now put the tombstone
    putTombstone(base, now, null);
    assertIsTombstone(base);

    + // todo create a raw fs for checking
    + S3GuardFsck fsck = new S3GuardFsck(rawFs, ms);

    /*- --------------------------------------------*/
    /* Begin S3FileSystem.finishedWrite() sequence. */
    /* ---------------------------------------------*/
    AncestorState ancestorState = getDynamoMetadataStore()
        .initiateBulkWrite(BulkOperationState.OperationType.Put,
            childPath);

Add new test: org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardDDBRootOperations#test_070_run_fsck_on_store

  @Test
  public void test_070_run_fsck_on_store() throws Throwable {
    // todo create a raw fs
    S3AFileSystem rawFs = ;
    S3GuardFsck s3GuardFsck = new S3GuardFsck(rawFs, metastore);
  }