Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13980 S3Guard CLI: Add fsck check and fix commands
  3. HADOOP-16502

Add fsck to S3A tests where additional diagnosis is needed

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • fs/s3
    • None

    Description

      Extend org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore#testPruneTombstoneUnderTombstone

          // the child2 entry is still there, though it's now orphan (the store isn't
          // meeting the rule "all entries must have a parent which exists"
          getFile(child2);
      
          + // todo create a raw fs
          + S3GuardFsck fsck = new S3GuardFsck(rawFs, ms);
      
          // a full prune will still find and delete it, as this
          // doesn't walk the tree
          getDynamoMetadataStore().prune(PruneMode.ALL_BY_MODTIME,
              now + MINUTE);
      

      Extend org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore#testPutFileDeepUnderTombstone:

          // now put the tombstone
          putTombstone(base, now, null);
          assertIsTombstone(base);
      
          + // todo create a raw fs for checking
          + S3GuardFsck fsck = new S3GuardFsck(rawFs, ms);
      
          /*- --------------------------------------------*/
          /* Begin S3FileSystem.finishedWrite() sequence. */
          /* ---------------------------------------------*/
          AncestorState ancestorState = getDynamoMetadataStore()
              .initiateBulkWrite(BulkOperationState.OperationType.Put,
                  childPath);
      

      Add new test: org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardDDBRootOperations#test_070_run_fsck_on_store

      
        @Test
        public void test_070_run_fsck_on_store() throws Throwable {
          // todo create a raw fs
          S3AFileSystem rawFs = ;
          S3GuardFsck s3GuardFsck = new S3GuardFsck(rawFs, metastore);
        }
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            gabor.bota Gabor Bota
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: