[HADOOP-18157] S3AFileSystem cannot find a fake dir on minio

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 3.2.2, 3.3.1
• Component/s: fs/s3

Environment

minio

• Assignee: Unassigned
• Reporter: JacobZheng
• Created: 09/Mar/22 04:06
• Updated: 29/Mar/22 08:34
• Resolved: 10/Mar/22 13:55

Description

I am using spark and delta lake for reading and writing datasets. After I upgraded the hadoop version, I found that writing files reported an error, indicating that the folder does not exist.

    java.io.FileNotFoundException: No such file or directory: s3a://table_cache/t5ffe773c987844cbba0aafd/_delta_log
        at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3356)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3185)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:3053)
        at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1760)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.exists(S3AFileSystem.java:4263)
        at org.apache.spark.sql.delta.storage.S3SingleDriverLogStore.listFromInternal(S3SingleDriverLogStore.scala:120)
        at org.apache.spark.sql.delta.storage.S3SingleDriverLogStore.exists(S3SingleDriverLogStore.scala:156)
        at org.apache.spark.sql.delta.storage.S3SingleDriverLogStore.write(S3SingleDriverLogStore.scala:174)
        at org.apache.spark.sql.delta.OptimisticTransactionImpl.doCommit(OptimisticTransaction.scala:742)
...

When this folder has been created, the error is still reported.

The related commit is 49f8ae96, It delete some code that dealing with Fake directory.

// Either a normal file was not found or the probe was skipped.
      // because the key ended in "/" or it was not in the set of probes.
      // Look for the dir marker
      if (probes.contains(StatusProbeEnum.DirMarker)) {
        String newKey = maybeAddTrailingSlash(key);
        try {
          ObjectMetadata meta = getObjectMetadata(newKey);

          if (objectRepresentsDirectory(newKey, meta.getContentLength())) {
            LOG.debug("Found file (with /): fake directory");
            return new S3AFileStatus(Tristate.TRUE, path, username);
          } else {
            LOG.warn("Found file (with /): real file? should not happen: {}",
                key);

            return new S3AFileStatus(meta.getContentLength(),
                    dateToLong(meta.getLastModified()),
                    path,
                    getDefaultBlockSize(path),
                    username,
                    meta.getETag(),
                    meta.getVersionId());
          }
        } catch (AmazonServiceException e) {
          if (e.getStatusCode() != SC_404 || isUnknownBucket(e)) {
            throw translateException("getFileStatus", newKey, e);
          }
        } catch (AmazonClientException e) {
          throw translateException("getFileStatus", newKey, e);
        }
      }
    }

Issue Links

• duplicates: HADOOP-18019 S3AFileSystem.s3GetFileStatus() doesn't find dir markers on minio