"info threads" shows that there are only 3 threads running:
- guac_user_input_thread (the thread segfault occured in)
- guacd_user_thread waiting for guac_user_input_thread
Note that there is no client thread running.
So here is what happenned: guac_rdp_client_thread exited and freed rdp_client->keyboard structure, while user input thread was still running and tried to access rdp_client->keyboard after it was freed.
This is an old bug. Found a 2-year old
GUACAMOLE-433 which looks exactly the same, but it was closed as "Cannot reproduce". It does not happen often indeed. We have 120 users actively using Guacamole (since COVID-19) and have seen this segfault only twice within the last month.
After some research I managed to reliably reproduce it.
It is reproducable with 1.1.0, staging/1.2.0 and master branches.
Steps to reproduce:
- Connect to RDP server
- Block connnection to RDP server with iptables (or create conditions for disconnect in any other way):
- Use keyboard or mouse actively. The easiest way is to push some key and hold it down for 30-60 seconds.
- Segmentation fault will happen in next 30-60 seconds at RDP disconnect time.
- Release the key you kept pressing
Segmentation fault can happen in multiple places depending on what you were doing at the time of disconnect, but typically it happens in keyboard or mouse handlers.
Since it is a race between threads, you may have to make several attempts to reproduce it.
I found it easily reproducable on a VM with a single CPU core, but on a physical machine with 24 CPU cores I had to make 3 or 4 attempts.