In the PDF Reference, we can read :
"... PDF can be entirely represented using byte values corresponding to the visible printable subset of the ASCII character set, plus white space characters such as space, tab, carriage return, and line feed characters. ASCII is the American Standard Code for Information Interchange, a widely used convention for encoding a specific set of 128 characters as binary numbers. However, a PDF file is not restricted to the ASCII character set; it can contain arbitrary 8-bit bytes,..."
So there are no recommended Charset... However instead of UTF-8, the default one should be US-ASCII or ISO-8859-1.
The problem comes from the comment line containing at least 4 binary characters (code >= 128) that comes just after the header line. As far as I remember, to match binary characters in JavaCC we must describe them using the Unicode notation (\uxxxx). With the charset CP1252, the character <9F> can't match with the token BINARY([\u0080-\u00FF]), because it is linked with the unicode character \u0178. (See http://unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1252.TXT)
So we have 3 possibilities :
 - Find a way to specify binary charaters without unicode notation in JavaCC
 - Add all unicode exceptions for the Cp1252 in the Binary token description
 - Update the BINARY token with [\u0080-\uFFFF] to avoid others charset specificities.
I prefer the first one, but if we can't do it maybe the third one will be the best to avoid further issues.
With following encodings, I run all my test set with the third option successfully :