I made some comparisons against building a large project using the distributed builds and regular builds. Below are the data I've gathered:
Distributed builds (with build agent installed in the same machine as the master):
1. using DB - 24 min 16 sec (build fresh); 12 min 40 sec
2. using LDAP - 33 min 1 sec (build fresh)
1. using DB - 23 min 35 sec (build fresh); 12 min 52 sec
2. using LDAP - 36 min 6 sec (build fresh)
I observed that the varying time of duration of the builds are caused mainly by the time it takes to checkout the project (when Build Fresh is enabled) and not necessarily by the build time. If you notice in the data above, the build duration time when it is not a fresh checkout are almost the same.
Using LDAP also makes the build slower because of the remote lookups for the authorization and permission.