Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15226 Über-JIRA: S3Guard Phase III: Hadoop 3.2 features
  3. HADOOP-13756

LocalMetadataStore#put(DirListingMetadata) should also put file metadata into fileHash.

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-beta1
    • Fix Version/s: 3.2.0
    • Component/s: fs/s3, test
    • Labels:
      None

      Description

      LocalMetadataStore#put(DirListingMetadata) only puts the metadata into dirHash, thus all FileStatus s are missing from LocalMedataStore#fileHash(), which makes it confuse to use.

      So in the current way, to correctly put file status into the store (and also set authoriative flag), you need to run

      List<PathMetadata> metas = new ArrayList<PathMetadata>();
      boolean authorizative = true;
      for (S3AFileStatus status : files) {
         PathMetadata meta = new PathMetadata(status);
         store.put(meta);
      }
      DirListingMetadata dirMeta = new DirMeta(parent, metas, authorizative);
      store.put(dirMeta);
      

      Since solely calling store.put(dirMeta) is not correct, and calling store.put(dirMeta); after putting all sub-file FileStatuss does the repetitive jobs. Can we just use a put(PathMetadata) and a get/setAuthorative() in the MetadataStore interface instead?

        Attachments

          Activity

            People

            • Assignee:
              gabor.bota Gabor Bota
              Reporter:
              eddyxu Lei (Eddy) Xu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: