Generated RTF files from fop don't work on TextEdit.app or Pages.app. When I try to open them i get an error that says "The document “helloworld.rtf” could not be opened. The file is not in the right format. The file might be corrupted, truncated, or in a different format than you expect."

I edited the rtf and narrowed it down to the font declaration section.

FOP sample font declaration:
{\fonttbl; {
\f0 Arial}{
\f1 Symbol}{
\f2 Times New Roman}{
\f3 sans-serif}
}
(for a simple document)

TextEdit.app sample font declaration:
{\fonttbl\f0\fswiss\fcharset0 Helvetica;}

The issue is the ";" after "fonttbl". When I change the FOP declaration to this it works fine in TextEdit.app:
{\fonttbl {
\f0 Arial}{
\f1 Symbol}{
\f2 Times New Roman}{
\f3 sans-serif}
}

I don't know anything about how the RTF spec works, so this may be different in larger documents.