[FLINK-32392] Several jobs failed on AZP with No space left on device

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.18.0, 1.16.3, 1.17.2
• Fix Version/s: 1.18.0, 1.16.3, 1.17.2
• Component/s: Test Infrastructure
• Labels: pull-request-available test-stability
• Assignee: Qingsheng Ren
• Reporter: Sergey Nuyanzin
• Created: 20/Jun/23 10:00
• Updated: 28/Jun/23 06:10
• Resolved: 28/Jun/23 06:10

Description

This Build failed with no space left https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=50162&view=logs&j=585d8b77-fa33-51bc-8163-03e54ba9ce5b

##[error]Unhandled exception. System.IO.IOException: No space left on device : '/home/vsts/agents/3.220.5/_diag/Worker_20230619-021757-utc.log'
   at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
   at System.IO.Strategies.BufferedFileStreamStrategy.FlushWrite()
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.Diagnostics.TextWriterTraceListener.Flush()
   at Microsoft.VisualStudio.Services.Agent.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id) in /home/vsts/work/1/s/src/Microsoft.VisualStudio.Services.Agent/HostTraceListener.cs:line 151
   at Microsoft.VisualStudio.Services.Agent.HostTraceListener.TraceEvent(TraceEventCache eventCache, String source, TraceEventType eventType, Int32 id, String message) in /home/vsts/work/1/s/src/Microsoft.VisualStudio.Services.Agent/HostTraceListener.cs:line 81
   at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message)
   at Microsoft.VisualStudio.Services.Agent.Util.ProcessInvoker.ProcessExitedHandler(Object sender, EventArgs e) in /home/vsts/work/1/s/src/Agent.Sdk/ProcessInvoker.cs:line 496
   at System.Diagnostics.Process.OnExited()
   at System.Diagnostics.Process.RaiseOnExited()
   at System.Diagnostics.Process.CompletionCallback(Object waitHandleContext, Boolean wasSignaled)
   at System.Threading._ThreadPoolWaitOrTimerCallback.WaitOrTimerCallback_Context_f(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading._ThreadPoolWaitOrTimerCallback.PerformWaitOrTimerCallback(_ThreadPoolWaitOrTimerCallback helper, Boolean timedOut)
   at System.Threading.PortableThreadPool.CompleteWait(RegisteredWaitHandle handle, Boolean timedOut)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
,##[error]The hosted runner encountered an error while running your job. (Error Type: Failure).

for 1.16, 1.17 it happens while 'Upload artifacts to S3'
for 1.18 while 'Deploy maven snapshot'

Issue Links

• links to: GitHub Pull Request #22858