When running the REEF.NET tests, the Windows firewall kicks in several times asking for Evaluator.exe to have permission to open a port. These popups are part of the application firewall of Windows: Whenever an unknown app wants to open a port, Windows asks for the users permission.
We don't see these for the JVM tests, as we probably all have given the java.exe that permission a long time ago, opening a gaping whole in our safety net from java-based malware
We can't do the same for the Evaluator.exe, as we create a new one on every build, and the Windows firewall is to distinguish them as new.
According to this thread on stackoverflow, we can however get a much cleaner solution: As long as we configure all our processes to listen on 127.0.0.1 only, the firewall won't be active.
The recent addition of the PortProvider and LocalAddressProvider classes should make that feasible.