Apparently there's a huge brain-fart in Apples interface for signing up external testers:
Name and Lastname are listed as "optional", but if you leave them out, it looks like the invite is not being send (or maybe the mail gets formatted in a way so that it bounces for some recipients, I don't know, but after trying two different e-mails 5 times each with no luck, it finally worked for both when I added name and lastname).