fail2ban fails to ban IP with “already banned”

Question

I enabled the dovecot plugin on my server and after a while (read: 10 seconds), I got my first ban:

2020-12-20 08:05:22,349 fail2ban.actions        [1495]: NO