[IO-785] FileUtils.deleteDirectory fails to delete directory on Azure AKS

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.9.0
• Component/s: Utilities

Environment

Azure Files Container Storage Interface (CSI) driver in Azure Kubernetes Service (AKS)
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"azure-aks-test-cluster-file-storage-class"},"mountOptions":["dir_mode=0777","file_mode=0777","uid=0","gid=0","mfsymlinks","cache=strict","actimeo=30"],"provisioner":"kubernetes.io/azure-file"}
storageclass.kubernetes.io/is-default-class: "false"
creationTimestamp: "2022-01-01T0-00:00:00Z"
name: azure-aks-test-cluster-file-storage-class
resourceVersion: "12768518"
uid: bc6-invalid-8c
mountOptions:

• dir_mode=0777
• file_mode=0777
• uid=0
• gid=0
• mfsymlinks
• cache=strict
• actimeo=30
  provisioner: kubernetes.io/azure-file
  reclaimPolicy: Delete
  volumeBindingMode: Immediate

• Assignee: Unassigned
• Reporter: Ivica Loncar
• Created: 13/Jan/23 09:04
• Updated: 21/Oct/24 12:13

Description

On Azure AKS file persistent volume (https://learn.microsoft.com/en-us/azure/aks/azure-files-csi) we've got following exception:

 
org.apache.commons.io.IOExceptionList: work/bef4a1a575c54ac099816b6babf4bde9/job-3418
	at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:330)
	at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1191)
	at com.xebialabs.xlrelease.remote.executor.k8s.KubeService.cleanWorkDir(KubeService.scala:107)
	at com.xebialabs.xlrelease.remote.executor.k8s.KubeJobExecutorService.cleanup(KubeJobExecutorService.scala:27)
	at com.xebialabs.xlrelease.remote.runner.JobRunnerActor.$anonfun$handleEvent$4(JobRunnerActor.scala:219)
	at com.xebialabs.xlrelease.remote.runner.JobRunnerActor.$anonfun$handleEvent$4$adapted(JobRunnerActor.scala:218)
	at scala.Option.foreach(Option.scala:437)
	at com.xebialabs.xlrelease.remote.runner.JobRunnerActor.com$xebialabs$xlrelease$remote$runner$JobRunnerActor$$handleEvent(JobRunnerActor.scala:218)
	at com.xebialabs.xlrelease.remote.runner.JobRunnerActor$$anonfun$receiveRecover$1.$anonfun$applyOrElse$2(JobRunnerActor.scala:45)
	at com.xebialabs.xlrelease.remote.runner.JobRunnerActor$$anonfun$receiveRecover$1.$anonfun$applyOrElse$2$adapted(JobRunnerActor.scala:45)
	at scala.Option.foreach(Option.scala:437)
	at com.xebialabs.xlrelease.remote.runner.JobRunnerActor$$anonfun$receiveRecover$1.applyOrElse(JobRunnerActor.scala:45)
	at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:35)
	at akka.event.LoggingReceive.apply(LoggingReceive.scala:96)
	at akka.event.LoggingReceive.apply(LoggingReceive.scala:70)
	at akka.persistence.Eventsourced$$anon$2$$anonfun$1.applyOrElse(Eventsourced.scala:643)
	at akka.actor.Actor.aroundReceive(Actor.scala:537)
	at akka.actor.Actor.aroundReceive$(Actor.scala:535)
	at com.xebialabs.xlrelease.remote.runner.JobRunnerActor.akka$persistence$Eventsourced$$super$aroundReceive(JobRunnerActor.scala:22)
	at akka.persistence.Eventsourced$$anon$3.stateReceive(Eventsourced.scala:771)
	at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245)
	at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244)
	at com.xebialabs.xlrelease.remote.runner.JobRunnerActor.aroundReceive(JobRunnerActor.scala:22)
	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579)
	at akka.actor.ActorCell.invoke(ActorCell.scala:547)
	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)
	at akka.dispatch.Mailbox.run(Mailbox.scala:231)
	at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)
Caused by: java.io.IOException: Cannot delete file: work/bef4a1a575c54ac099816b6babf4bde9/job-3418/input
	at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1343)
	at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:323)
	... 32 common frames omitted
Caused by: org.apache.commons.io.IOExceptionList: work/bef4a1a575c54ac099816b6babf4bde9/job-3418/input
	at org.apache.commons.io.file.PathUtils.setReadOnly(PathUtils.java:944)
	at org.apache.commons.io.file.PathUtils.deleteFile(PathUtils.java:485)
	at org.apache.commons.io.file.PathUtils.delete(PathUtils.java:392)
	at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1340)
	... 33 common frames omitted
Caused by: java.nio.file.FileSystemException: work/bef4a1a575c54ac099816b6babf4bde9/job-3418/input: Operation not supported
	at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
	at java.base/sun.nio.fs.LinuxDosFileAttributeView.updateDosAttribute(Unknown Source)
	at java.base/sun.nio.fs.LinuxDosFileAttributeView.setReadOnly(Unknown Source)
	at org.apache.commons.io.file.PathUtils.setReadOnly(PathUtils.java:919)
	... 36 common frames omitted

Deleting a directory using PathUtils and empty option list works, but is was not obvious what will happen.