When the directive "AddHandler cgi-script cgi" is used, and presumably similar constructions, matches will be made on filenames containing ".cgi" and not just files with the extension ".cgi" (or .pl, or other). This was discovered when a CGI program was compiled from source named "compiled.cgi.c" and the executable output was "compiled.cgi". When both are visible on an index page delivered from Apache, clicking on "compiled.cgi.c" should cause it to deliver a plain text document. Instead, Apache attempts to run it as a cgi program. The compilation itself runs as expected because Apache properly delivers "compiled.cgi" to the handler. Somewhere (mod_mime ?) an improper match is made on file extensions. I couldn't find it myself.
An "extension" in apache is any period-delimited part of the filename after the first period. So in one.two.three.four, all of two, three and four are extensions. This is documented in the "Files with multiple extensions" section of http://httpd.apache.org/docs-2.0/mod/mod_mime.html and at http://httpd.apache.org/docs-2.0/mod/directive-dict.html#Syntax under "extension".
Joshua Slive points out something missed. The solution is to use the pattern matching capabilities of Apache: <Files ~ "\.cgi$"> SetHandler cgi-script </Files> instead of the "AddHandler cgi-script .cgi" as in the default configuration file.
Note that using the regular expression "\.cgi$" to identify CGI scripts has the potentially unintended consequence of employing the cgi-script handler on the hidden file ".cgi".