Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13980 S3Guard CLI: Add fsck check command
  3. HADOOP-16502

Add fsck to S3A tests where additional diagnosis is needed

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: fs/s3
    • Labels:
      None
    • Target Version/s:

      Description

      Extend org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore#testPruneTombstoneUnderTombstone

          // the child2 entry is still there, though it's now orphan (the store isn't
          // meeting the rule "all entries must have a parent which exists"
          getFile(child2);
      
          + // todo create a raw fs
          + S3GuardFsck fsck = new S3GuardFsck(rawFs, ms);
      
          // a full prune will still find and delete it, as this
          // doesn't walk the tree
          getDynamoMetadataStore().prune(PruneMode.ALL_BY_MODTIME,
              now + MINUTE);
      

      Extend org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore#testPutFileDeepUnderTombstone:

          // now put the tombstone
          putTombstone(base, now, null);
          assertIsTombstone(base);
      
          + // todo create a raw fs for checking
          + S3GuardFsck fsck = new S3GuardFsck(rawFs, ms);
      
          /*- --------------------------------------------*/
          /* Begin S3FileSystem.finishedWrite() sequence. */
          /* ---------------------------------------------*/
          AncestorState ancestorState = getDynamoMetadataStore()
              .initiateBulkWrite(BulkOperationState.OperationType.Put,
                  childPath);
      

      Add new test: org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardDDBRootOperations#test_070_run_fsck_on_store

      
        @Test
        public void test_070_run_fsck_on_store() throws Throwable {
          // todo create a raw fs
          S3AFileSystem rawFs = ;
          S3GuardFsck s3GuardFsck = new S3GuardFsck(rawFs, metastore);
        }
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              gabor.bota Gabor Bota
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: