What you've done looks pretty good and is basically one of two standard architectures that I see a lot.
Mixed feelings about:
Fine, we have a pluggable repository implementation, but how often do you really have different implementations of the same persistence interface?
Pluggable is often touted as being good design but I've never once seen a team swap out a major implementation of something for something else. They just modify the existing thing. IMHO "pluggability" is only useful for being able to mock components for automated unit testing.
I am not sure if the UI should only communicate with the Business Layer via View models, or should I use Domain Models to transfer data, as I do now. For display, I am using view models, but for data transfer I am using Domain models. Wrong?
I reckon view models are a UI (MVC Web) concern, if you added a different type of UI for example it might not require view models or might need something different. So I think the Business layer should return domain entities and allow them to be mapped to view models in the UI layer.
What I don't like:
The Core project is now referenced in every other project - because I want/have to access the Domain models. In classic Onion architecture, the core is referenced only by the next layer.
As others have said this is quite normal. Usually everything ends up having a dependency on the Domain.
The DbContext is implemented in the .Core project, because it is being generated by the Entity Framework, in the same place where the .edmx is. I actually want to use the .EDMX for the visual model design, but I feel like the DbContext belongs to the Persistence layer, somewhere within the database-specific repository implementation.
I think this is a consequence of Entity Framework. If you used it in "Code First" mode you actually can - and usually do - have the context and repository in the Persistance layer with the Domain (represented as POCO classes) in what you've called Core.
As a final question - what is a good architecture which is not over-engineered (such as a full-blown Onion, where we have injections, service locators, etc) but at the same time provides some reasonable flexibility, in places where you would realistically need it?
As I touched on above I wouldn't worry about the need to swap things out except to allow for automated unit tests. Unless there is a specific requirement you know about that will make this very likely.
Good luck!