JPA maintains object identity and will not persist an existing object.
Change you code to be correct,
for(int i=1; i <= 10; i++) {
Department department = new Department("Test Department");
Employee e = new Employee("EMP" + i, department);
em.persist(e);
}