When BufferedBlockMgr blocks must be flushed to disk but the write cannot be completed due to lack of disk space on the scratch disks, queries fail with a cancelled status and no further information. Queries should instead fail with an exception status and an error message indicating the scratch disk was full.
This can be reproduced by setting the scratch dir to a disk with negligible space and forcing a query to spill to disk. The way to determine this occurred is by looking for the following in the impalad log:
I0831 14:13:35.791662 14318 buffered-block-mgr.cc:769] Query: f74317d168f9d915:58d67871e44fa2a2 write complete callback with error.
I0831 14:13:35.791698 14318 runtime-state.cc:225] Error from query f74317d168f9d915:58d67871e44fa2a2: posix_fallocate(279, 17095983104, 8388527) failed for file /tmp/impala-scratch/f74317d168f9d915:58d67871e44fa2a2_747758b8-ff16-4a00-8d1f-0566ab71c787 with returnval=28 description=Error(11): Resource temporarily unavailable
There is no workaround.