Yes. In the apps dashboard there is a section called Role. There you can add a new user as admin and remove the current one. The previous user will no longer be able to make changes in the app.
- Goto https://developers.facebook.com/apps/
- Select you application.
- Click Edit App
- Click Roles
- Add new user as admin
- Remove the current one.