Unmatched messages get an unmatched ack, however this ack is ignored by the topic message store under the assumption that there will be some subsequent ack that will more the cursor forward and allow cleanup to happen. However this may never happen and messages can accumulate till that subscription is removed.
The jdbc topic message store needs to track the unmatched acks in the normal way, however the ack sequence id is linear and any gap in unmatched would potentially ack previous messages of the same priority.
The best we can do without converting to an individual ack table is the have expiry kick in and remove/ack these messages in the normal way.
The caveat being that expiry processing needs to be relatively linear in time. The broker attribute enableMessageExpirationOnActiveDurableSubs enables this feature, off by default.