I want to create a new user group.
The members in this user group can create new users and new groups.
I think creating users
You cane manage that through Alfresco Share. In your Alfresco installation directory:
/tomcat/webapps/alfresco/WEB-INF/classes/alfresco/model/permissionDefinitions.xml
you'll find the permissionDefinitions.xml file. Through that file you can manage your custom roles for Alfresco users and groups.
Here, in this WIKI you can find some hints about what you need to change to accomplish that.
Hope it helps. The wiki helps here, if you need some better explanation please let me know.
You could add a custom role in permissionDefinitions.xml
but I don't think that would help in this instance, since the ability to create users and groups plus the other administrative actions are hard-wired to the built-in administrators role.
Instead, you could create a custom web script which performs a check to see if the current user is a member of your custom security group. Since the authenticated user may not be a repository admin (and probably isn't) then you would need to use something like this Sudo tool for Alfresco webscripts to temporarily give them permission to create the objects.
You would need to be competent in Java webscript development in order to implement this. There are some good resources on the Alfresco wiki, documentation and training materials available online.