The bug is illustrated by the following code snippet:
With auto-commit off, add duplicates into a deferred primary key. Then set a savepoint, drop the table, and roll back to the savepoint.
Apparently, when you drop the table, information about any constraint violations seen on that table is lost, and that information is not restored when the drop table operation is undone by the rollback to savepoint.
So when you commit the transaction after having rolled back the drop operation, no deferred checking of constraints happens, and the duplicates you have inserted are committed.